jayfoxrox / xbox-tools Goto Github PK

Should be in readme (formatted differently?):

pacman -S mingw-w64-x86_64-python3

For optional stuff:

pacman -S mingw-w64-x86_64-python3-pip

For the screenshot script:

pacman -S mingw-w64-x86_64-python3-pillow

For realtime audio output:

pacman -S mingw-w64-x86_64-portaudio
python3 -m pip install pyaudio

For running:

XBOX=192.168.178.158 python3 ./screenshot.py

When running python-scripts on XQEMU we might want to slow down the timing critical code (such as audio buffer streaming).
This could be done by providing our own time functions which respect a virtual clock of the Xbox.
A real Xbox would just forward this to wall-clock-time.

It only works sometimes. Also it's not yet known what the bootrom does, there seems to be a lot of strange stuff happening during reset.

Physical memory read commands should automaticly handle to mapping the buffers into viewable space.
All weird |= 0x80000000 should be removed.

It took me a while to figure out what's wrong with my code, so I used various loop variable names.
This should be fixed and they should be replaced with more meaningful names

5849 [or something?] works in XQEMU // 5933 on hardware doesn't

The buffer description looks fine (same for both). I'm not sure if this is due to hw or sw differences.
The colors in the broken screenshots are still correct.
This might be related to memory mapping or the GPU swizzling data or something?

Due to how the kernel is loaded on Xbox, we can currently (?) not dump the entire kernel (without the MCPX key).
The dumped xboxkrnl.exe will miss the INIT section (in most cases), which creates a kernel which is runnable from a savestate / on initialized hardware. However, the kernel will be unable to initialize the hardware and boot the system.

I came up with an attack to dump the INIT section seperately on a cold-boot (as the memory location of INIT is known and it's still in memory after free()). However, gaining access to be the first XBE to run after a cold-boot is tricky, maybe even impossible without hardware hacks.
Also, if hardware is involved this tool failed it's purpose and we could also just dump the MCPX ROM directly.

= We need new attacks to dump MCPX ROM or the full Kernel image.

.

The following tests should ideally be python scripts to control the GPU

Anti Aliasing

This is required to fix the scaling issues related to Anti-Aliasing

Vertex Transform Program / D3D State Shader test

I require state shader code so we can compare our shaders against the xbox shaders

Vertex Transform Program / D3D Write Shader test

I have to test if writing to constants will change them accross the entire draw call or even hardware

Shader constant test

We have to read back shader constants and see which are hardware controlled (also see 'nv2a-re')

Texture format test

We have to upload garbage to the GPU in every available texture format to see if xqemu produces the same garbage

Draw call and vertex attribubte testing

We have to upload various type of primitives using differet upload methods to see what's influenced

Fog testing

We need examples for fog to test the fog combiner input and vertex output

Video overlay test

we need a tool to test video overlay with various keycolor and scaling options

Currently the path to a56 is hardcoded (to a local path on my drive). Should be fixed asap.

However, I'm not sure how to best go about this.
I think we should look in os.environ['A56'] and if that does not exist in the current path.
If nxdk includes a56 in the future it could also be located through os.nviron[NXDK_DIR] in the future.

I'd expect that there is a python package / module / lib to do this for us.

I am trying to use the dump-xbox utility to get the eeprom and cert keys from my xbox. I managed to compile it with nxdk and upload it to my xbox but when I try to execute it, my keys appear as a long string of 02X (the same hex present in the code). I also noticed that the variables errors and MB of RAM show up as "i" instead of numbers.
Do you know what could I be doing wrong? I attach the compilation log below. Also, may I ask if you can release the compiled xbe? There shouldn't be any legal issues if it is compiled with nxdk instead of the MS xdk.

This tool is best ignored. There'll be a new tool to replace this soon, which approaches this from a different angle.

Code contains this:

  mode = val & 0x000E0000
  if (mode & 0x00010000):
    print("  HDTV 480i")

but also this:

  if val & 0x00010000:
    print("  Widescreen")

Both of them check val & 0x00010000, although it's not even reachable in the mode variable, as that particular bit was masked. Therefore, it is assumed that the resolution field is bad.

• audio.py should be ac97 or aci.py
• dsp.py should be apu.py

There should be a backend for nxdk-rdt.

I originally intended to have most of these scripts in nxdk-rdt, but then had to use QEMU gdb for the APU (as nxdk does not have APU support).
Even for working on the hardware (where we don't have gdb) we can't use nxdk-rdt because we still don't have APU support. To solve that issue I added a XBDM backend (which is pretty horrible as it requires a hook).

I still feel we should have an nxdk-rdt backend for working with the GPU / running unit tests.
However, most base scripts should stay in this repo as they must also support various other backends too (so they can be re-used more easily). Moving the scripts away would mean a loss of control.

The current header is not allowed to use more than 2 channels!

nxdk-rdt is still horribly broken so it has a hack to split larger writes into smaller chunks.
The hack should be removed once nxdk-rdt has been improved. See https://github.com/mborgerson/nxdk-rdt/issues/5

Additionally the play_ac97.py can only be ran once as a result of this (further writes will break the nxdk-rdt network code).

Generally add more error checking.

The tool has not been tested on exploited files. Those should be detected, possibly even dumping the exploit code.

The size of each glyph is controlled by the scale variable and it's not well adjusted.
The font appears smaller or larger than other fonts.

The missing-glyph also has a fixed size which is not affected by the scale variabe.

Additionally, rounding issues will result in some gaps between triangles.

Death to all tabs.
I only used tabs as that is what nxdk-rdt uses.

The original concept for this did not work out. This needs further investigation.

The idea was:

• Pause the puller (= pusher comes to halt as there is no free space)
• Pause the pusher (= disable CACHE being overwritten)
• Set pusher address as puller address (= puller has nothing to do)
• Unpause the puller (= puller still has nothing to do)
• Set pusher address to new target puller address (= puller runs until pusher address)

Once the fake pusher address reaches the real pusher address (= no more data):

• Pause the puller (= FIFO is idle now)
• Assert that pusher address is at real pusher address (= need to continue writing at proper address)
• Unpause pusher (= let pusher fill CACHE)
• Pause pusher (= pusher should have advanced / filled CACHE)

Now the initial procedure can be repeated from the second step

The issue, however, is that even with the pusher paused and pusher address = puller address, the entire CACHE seems to be filled with the same method / data pair.
This can also have implications for the general pausing of the NV2A so that should be fixed / cleared up asap!

There should be a KD backend.

To my knowledge, there is no existing KD client lib for Python.

There is '/scripts/windpl/windpl.pl' in XQEMU.
If a backend for KD is written, it should be seperated from this project (and offered as a Python module).

(There should still be a memory_kd.py but it should be minimal and use the then-existing Python KD interface)

Needs decompression.

My Xbox one x eject button doesn’t respond anymore. No sounds. No ejects. Nothing. Is there a script that can do this for me? Or do I need to take it to Microsoft?

https://github.com/JayFoxRox/xbox-tools/blob/master/dump-xbox/dump-xbox.c#L76

I don't know what my motivation was for this. I assume I had to do it because nxdk does not support libc file-io properly.

Causes issues brought up in #34

Script should be cleaned up and dump out SS.bin, DMI.bin and PFI.bin, as said by the comment at the top of the file.

Commands to locking and unlocking the DVD should be added (seperate script or command line option?)

From the README:

Dumping the secret southbridge ROM (including the RC4 Key) is very hard and not possible (yet?).

This is not factually true. Ofcourse the MCPX is dumpable. Just not through homebrew / these tools.

Touched by #40 , but should be put in simpler terms to describe the overall expected result: filtering and hooking NV2A pushbuffers on CACHE level

The memory allocations done using the ke module should be tracked, either by the API or by the scripts.
This affects play_ac97.py (and possibly others) which allocate a lot of memory but never free it.

When using nxdk-rdt in XQEMU, we need:

  # Set frame duration (?!)
  if True:
    apu.write_u32(NV_PAPU_SECTL, 3 << 3)

Only if this has been done, se_frame will be called.

I did not test what happens on real hardware.

The xbdm interface does not have this issue because it's likely running with DirectSound already initialized.

Currently it expects data at offset 56, this is ugly.
The converter should parse chunks properly.

The APU code was developed by debugging XQEMU. As XQEMU is also in control of doing the APU writes the script might not work on hardware.

Additionally the buffers will be probably be inaccessible on hardware due to the audio hardware accessing (and possibly also free'ing) them.

The README should be clear that this is open-source and accepting contributions.

The interface should be unified and use python mechanics [such as import] so the user is provided with a set of interface specific core functions (Memory access, Calling, ..) + functions build-on top of that (ACI, APU, GPU, Kernel, Paging, ..).

The configuration format is pretty bad. 2 of the available files are also not correct anymore.

The idea behind the format is to make it possible to have one json file per product or protocol.
The same files (or files generated from it) are supposed to be fed into xbox emulators later.
Ideally they'll even be used to generate the xboxdevwiki pages [or replace them].
This also means we'll need tools to dumb all the USB configuration descriptors and put it in each config.

.

XBDM currently uses getmem / setmem which transfers data as ASCII. For larger chunks this will be very slow too (more work + data conversion).

Additionally the memory hook for u8, u16 and u32 requires 3 network packets currently:

• setmem
• resume thread=workspace
• getmem

This can be changed by directly modifying the xbdm packet instead of communicating through a memory bank. The command could also be extended to read and write data of arbitrary length to solve all issues of XBDM.

This is critical for streaming buffers back to a debugger as valuable time passes during these reads!

It has various mostly unused codepath:

• Code is given in source
• Code is loaded from hex-string
• Code is loaded from tiny-gp.inc

This should probably be improved.

The last entry seems to be 0x1000 bytes off. Stupid "smart" list logic :(

G:\github.com\JayFoxRox\xbox-tools\xtf-converter>.\main.py "Xbox.xtf" >"Xbox.svg"
Traceback (most recent call last):
File "G:\github.com\JayFoxRox\xbox-tools\xtf-converter\main.py", line 70, in
print("<!-- Decoding symbol " + str(c) + ": "\" + symbol + "\" -->")
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0' in position 27: ordinal not in range(128)

Update 1:
Text left out of the message for the print function between the quotes;

Update 2 and 3:
Ah ha, figured out how to get the comment to show. Escape (\) the comment tag characters.

<!-- Decoding symbol " + str(c) + ": \"" + symbol + "\" -->

Update 3:
Using Pyhon 2.7.13

Update 4:
The backslashes are actually still in the print("... just they get stripped out when posting without escaping them too. Fixed now to show they are still in the print function.

While this is a nice idea, it only works with certain gamepads / controllers.
How exactly this should be done is not decided yet.

Something about how I import my modules (which is pretty ugly) does not fly with MSYS.

I'm an absolute python noob, so if anyone could fix this for me that'd be appreciated. It's really important for the development of XQEMU because I need report / script output from Windows users.

A lot of the python scripts will disappear soon as they move into seperate projects.
This is primarily as we have a good place to put these tools now where they can grow more easily (XboxDev).

The python-scripts/xbox folder has moved to the xboxpy module (with some API changes to come).

The xbox folder only remains until this repository has finished the transition to xboxpy (which also depends on the API stabilizing).
New tools should depend on xboxpy only. Changes involving xbox folder should be avoided. Refactor for use of xboxpy instead (and implement changes there).

While all Xbox ADPCM files are also IMA ADPCM files (I think), this is not true the other way around: IMA ADPCM might not be Xbox ADPCM files.
As such, the converter will struggle with them if the block alignment does not work properly.

As the files are otherwise working (once they pass header checks), a warning message should be shown, warning the user that they are feeding Xbox ADPCM which is tagged as IMA ADPCM.
To prepare users for failure, this message should be shown before anything else.

The code currently loops over all AC97 buffers, when the hard assumption for the logic is that there is only ever 1 buffer in existence.
If AC97 buffer looping is done it should be the inner, not the outer loop.

Trace length + output path should be an argument.

The overall code quality is poor too.

The base address of xbdm.dll is currently hardcoded.
This needs extensions in 'memory_xbdm.py' so it can use modules to find the base address. This mostly needs parsing of the received lines.

Needs to parse this:

I think a dictionary would be nice (with integers [most fields], strings [name] and bools [tls, xbe])
(And the return value would be a list of such dicts)

In fact, it's far off. However, audacity seems to load my file as float and not PCM S16.
I should compare output against ffmpeg etc. too.