# Tools for working with a net dimm

This repository started when I looked at triforcetools.py and realized that it was ancient (Python2) and contained a lot of dead code. So, as one of the first projects I did when setting up my NNC to netboot was to port triforcetools.py to Python3, clean up dead code and add type hints. I also added percentage display for transfer and `--help` to the command-line. The current code is capable of sending data to a net dimm with on-the-fly patches, managing a net dimm to ensure that it always gets a particular game every boot, reading information from the net dimm cartridge, editing EEPROM dumps and attaching both SRAM and EEPROM dumps to Naomi ROMs when sending images to net boot. This has all been tested on a Naomi with a net dimm, but has not been verified on Triforce/Chihiro. There is no reason why the non-EEPROM code should not work on both Triforce and Chihiro, however, as much of the information was verified by reverse-engineering both the net dimm 3.17 firmware and the SEGA transfergame.exe utility.

## Setup Requirements

This requires at least Python 3.6, and a few packages installed. To install the required packages, run the following. You may need to preface with sudo if you are installing into system Python. However, I recommend setting up a virtualenv instead of polluting your system environment.

```
python3 -m pip install -r requirements.txt
```

Remember to append a `--upgrade` to the above command if you are refreshing packages!

## Script Invocation

For all of these scripts, they should run out of the box on Linux and Mac once you've run the above installation command. However, on Windows you will have to prefix all commands with `python3`. For example, instead of running `./netdimm_send --help` you would instead run `python3 ./netdimm_send --help`. All scripts will function identically thereafter, regardless of the operating system, as long as you have Python 3.6 or greater and the correct dependencies installed.

### netdimm_info

This script requests firmware information from a single cabinet, displaying it on the screen. Included in that info is the net dimm firmware revision, the memory capacity of the net dimm, the available memory that can be used for game storage, the current game's CRC image, whether the net dimm thinks that CRC is valid or not and whether the CRC check screen has been disabled for this game or not. Invoke the script like so to see options:

```
./netdimm_info --help
```

Assuming your net dimm is at 192.168.1.1, the following will print information about the firmware running on the cabinet:

```
./netdimm_info 192.168.1.1
```

### netdimm_send

This script handles sending a single binary to a single cabinet, assuming it is on and ready for connections. For most setups, this means that the cabinet is sitting on the "CHECKING NETWORK" screen. In some cases, you can also send a game to an already-running net dimm and it will reboot the currently running game and receive a new one. This does not work for all games, however, as the reboot hook that the net dimm installs does not work for every game. Invoke the script like so to see options:

```
./netdimm_send --help
```

You can invoke it identically to the original triforcetools.py as well. Assuming your net dimm is at 192.168.1.1, the following will load the ROM named `my_favorite_game.bin` from the current directory:

```
./netdimm_send 192.168.1.1 my_favorite_game.bin
```

As well as sending a single game to a Net Dimm, this script can optionally handle applying patches. See `--help` for more details. It can also handle sending settings along with a game for the Naomi target. Again, see `--help` for more details. This can be a valid EEPROM or SRAM file as obtained from an emulator. Also see `edit_settings` for how to generate settings files that can be sent along with a game to a Naomi system. It can also disable the CRC check screen in order to boot the ROM you are sending faster. See `--help` for more information.

### netdimm_ensure

This script will monitor a cabinet, and send a single binary to that cabinet whenever it powers on. It will run indefinitely, waiting for the cabinet to power on before sending, and then waiting again for the cabinet to be powered off and back on again before sending again. If the cabinet is already running a game when it is run and that game matches what it was attempting to send, it will not reboot the cabinet but instead wait for the cabinet to be powered off and back on to send once more. If the cabinet is already running a game and that game does not match what it is attempting to send, it will reboot the cabinet and send the new game. If the battery in your net dimm is functional and your cabinet boots into the game that it was attempting to send, it will not resend the game but instead wait for the CRC to fail before resending. Invoke the script like so to see options:

```
./netdimm_ensure --help
```

It works identically to netdimm_send, except for it only supports a zero PIC and it tries its best to always ensure the cabinet has the right game. Run it the same way you would run netdimm_send. Just like `netdimm_send`, this script can optionally handle applying patches. See `--help` for more details. It can also handle sending settings along with a game for
the Naomi target. Again, see `--help` for more details. This can be a valid EEPROM or SRAM file as obtained from an emulator. Also see `edit_settings` for how to generate settings files that can be sent along with a game.

### netdimm_receive

This script handles querying the current running game's size and then downloading it to a file. It is fairly useless as it won't dump anything other than what was previously sent to the net dimm by `netdimm_send` or `netdimm_ensure`. However, it is included for documentation purposes. Invoke the script like so to see options:

```
./netdimm_receive --help
```

### netdimm_menu

This script creates a simple menu ROM given a directory containing Naomi ROM files, uploads it to a Naomi and then waits for a user selection before rebooting the cabinet and sending the selected ROM to the net dimm. Use this instead of a Raspberry Pi with an LCD screen to select a game on boot up. Note that it only works with Naomi at the moment! Invoke the script like so to see options:

```
./netdimm_menu --help
```

### netdimm_peekpoke

This script connects to a net dimm and requests it to peek at system RAM or poke a value into system RAM at a specified address. This is mostly provided as a curiosity as there are not many uses for such a utility. Invoke the script like so to see options:

```
./netdimm_peekpoke --help
```

### binary_patch

This script can either diff two same-length binaries and produce a patch similar to the files found in `patches/` or it can take a binary and one or more patch files and produce a new binary with the patches applied. Note that this is just a frontend to the same utility that lives in <https://github.com/DragonMinded/arcadeutils> and as such all documentation there applies here as well. The patches that this produces in diff mode can also be applied on-the-fly when using `netdimm_send` or `netdimm_ensure` by using the `--patch` argument to either tool. See either of the tools above for more information. Invoke the script like so to see options:

```
./binary_patch --help
```

To diff two binary files named `file1.bin` and `file2.bin`, outputting their differences to a file named `differences.binpatch`, run like so:

```
./binary_patch diff file1.bin file2.bin --patch-file differences.binpatch
```

To apply a patch to a binary file named `file.bin` with patches found in `differences.binpatch`, outputting it to a new file named `newfile.bin`, run like so:

```
./binary_patch patch file.bin newfile.bin --patch-file differences.binpatch
```

### rominfo

This script will read a ROM and output information found in the header. Currently it only supports Naomi ROMs but it can be extended to Triforce and Chihiro ROMs as well if somebody wants to put in the effort. Invoke the script like so to see options:

```
./rominfo --help
```

To output information about a particular binary, run like so:

```
./rominfo somefile.bin
```

### eeprominfo

This script will read an EEPROM and output the serial number as well as the contents of the game section as hex digits. Currently it only supports Naomi EEPROMs. Optionally it can load a settings definition file for the EEPROM out of the `settings/definitions/` directory and print the current settings contained in the EEPROM. This is most useful when reverse-engineering the settings format for a particular game. Invoke the script like so to see options:

```
./eeprominfo --help
```

To output information about an EEPROM file as extracted from demul's `nvram` folder, run like so:

```
./eeprominfo dummy.eeprom
```

### attach_sram

In 'attach' mode, this script will take an SRAM file from an emulator such as demul and attach it to an Atomiswave conversion game so that your Naomi initializes the SRAM with its contents. If an Atomiswave conversion ROM already has an SRAM initialization section, it will overwrite it with the new SRAM. Otherwise, it enlarges the ROM to make room for the init section. Use this to set up defaults for a game using the test menu in an emulator and apply those settings to your game for netbooting with chosen defaults.

In 'extract' mode, this script will extract a previously-attached SRAM from a Naomi game so that you can load it in an emulator such as demul. If the ROM does not have an attached SRAM section, it will not extract anything.

Invoke the script like so to see options:

```
./attach_sram --help
```

To attach a SRAM file from demul to a ROM named demo.bin, run like so:

```
./attach_sram attach demo.bin dummy.sram
```

### attach_settings

In 'attach' mode, this script will take an EEPROM file from an emulator such as demul and attach it to Naomi game so that your Naomi initializes the EEPROM with its contents. If a Naomi ROM already has an EEPROM initialization section, it will overwrite it with the new EEPROM. Otherwise, it enlarges the ROM to make room for the init section. Use this to set up defaults for a game using the test menu in an emulator and apply those settings to your game for netbooting with chosen defaults.

In 'extract' mode, this script will extract a previously-attached EEPROM from a Naomi game so that you can load it in an emulator such as demul. If the ROM does not have an attached EEPROM settings file, it will not extract anything.

In 'info' mode, this script will print information about any attached EEPROM settings on an existing ROM, displaying what version of the settings trojan was used, whether some options were selected and the full list of settings chosen for the game.

In 'edit' mode, this script will extract an existing EEPROM settings file from a ROM, let you edit those settings, and then save the settings back to the ROM. If the Naomi ROM does not have any EEPROM settings already attached, it will create them using defaults stored in the definition files before letting you edit those defaults and then save them.

Note that some Naomi games have protection code that prevents successful modification of the header. In these cases, attempting to apply settings to a ROM will result in the game freezing randomly. Usually this can be seen in the attract sequence. To circumvent this and apply settings successfully, check the `patches/` folder to see if a `noprotect` patch for that game exists and apply it first before attempting to attach or edit settings for a ROM.

Invoke the script like so to see options:

```
./attach_settings --help
```

To attach an EEPROM file from demul to a ROM named demo.bin, run like so:

```
./attach_settings attach demo.bin dummy.eeprom
```

To edit the settings you just attached, run like so:

```
./attach_settings edit demo.bin
```

### edit_settings

This script spawns a command-line EEPROM file editor. Use this to create a new EEPROM file from scratch or edit an EEPROM file that you've previously made using an emulator. Note that editing settings for an arbitrary game requires that the game have a settings definition file in `settings/definitions/`. Settings definitions are looked up using the game's serial number which is found both in the rom header for a game (use `rominfo` to view this) and in a previously-created EEPROM file (use `eeprominfo` to view this). The resulting edited EEPROM can be attached to a Naomi ROM using `attach_settings` so that the game will load those settings when it boots up. It can also be patched on-the-fly when sending a Naomi ROM using the `--patch` argument in either `netdimm_send` or `netdimm_ensure`. Invoke the script like so to see options:


```
./edit_settings --help
```

### Free-Play/No Attract Patch Generators

Both `make_freeplay_patch` and `make_no_attract_patch` can be invoked in the same manner and will produce a patch that applies either forced free-play or forced silent attract mode. Note that these patches are considered obsolete as you can customize all system settings using `attach_settings` or `edit_settings` as documented above. Still, they are provided for posterity assuming you wish to use another netboot package and just want to patch your ROMs to be in free-play. You can run them like so:

```
./make_no_attract_patch game.bin --patch-file game_no_attract.binpatch
./make_freeplay_patch game.bin --patch-file game_freeplay.binpatch
```

You can also see options available for running by running it with the `--help` option:

```
./make_no_attract_patch --help
./make_freeplay_patch --help
```

### Default Game Settings Patch Generator

The `patch_default_settings` script attempts to take an EEPROM file that was obtained from an emulator or created using `edit_settings` and apply the game section to a ROM such that when it initializes EEPROM it does so with your chosen settings instead of its original defaults. Note that this requires a settings definition file for the game in order to work properly and is not guaranteed to work. It operates under the principle that many games have the entire default game EEPROM compiled into their code somewhere and changing that changes what they write to the EEPROM when creating new settings. It uses the defaults specified in the settings definition file to figure out what EEPROM contents to search for and if it finds a chunk of data matching that contents it will patch the ROM to have the settings found in the specified EEPROM file instead. If you wish to generate a patch instead of patching the ROM itself, you can run with the `--patch-file` argument. Invoke the script like so to see options:

```
./patch_default_settings --help
```

## Web Interface

Along with scripts, several libraries and a series of patches, this repository also provides a simple web interface that can manage multiple cabinets. This is meant to be run on a server on your local network and will attempt to boot your net dimm-capable cabinet to the last game selected on every boot. It also allows you to easily select new games using a drop-down as well as configure settings for the Naomi platform and general patches to apply when sending a game to any platform. Use this if you want to treat your netboot setup much like a cabinet with a legitimate cartridge in it, but still allow yourself and friends to select a new game easily.

To try it out with the test server run the following:

```
./host_debug_server --config config.yaml
```

Like the other scripts, you can run this command with the `--help` flag to see additional options. By default, the config will look for ROMs in the `roms/` directory, patches in the `patches/` directory, SRAM files in the `srams/` directory and Naomi settings definitions files in the `naomi/settings/definitions` directory. It will listen on port 80. You do not want to run the above script to serve traffic on a production setup as it is single-threaded and will dump its caches if you change any files. By default, the server interface will hide advanced options, such as cabinet configuration. To show the options, either type the word 'config' on the home page, or go to the `/config` page. Both of these will un-hide the configuration options until you choose again to hide them.

### Production Setup

Since this uses a series of threads to manage cabinets in the background, it can be somewhat difficult to install in a normal nginx/uWSGI setup. A wsgi file is provided in the `scripts/` directory that is meant to be used alongside a virtualenv with this repository installed in it. Config files should be copied to the same directory, and don't forget to update the ROMs/patches/settings definition directories in your config to point at their actual locations. If you do something similar to my setup below, don't forget to set up a virtualenv in the venv directory!

Since the default configuration for this repository is to be used as a package in other code, setup.py will default to only installing the parts of the package that matter for that. If you want to use pip to update your virtualenv when running the full web interface you should run `FULL_INSTALLATION=1 pip install . --upgrade` to tell setup.py to install the web server packages as well. Make sure you do this with your virtualenv activated to avoid polluting your system packages!

My uWSGI config looks something similar to the following:

```
[uwsgi]
plugins = python3
socket = /path/to/wsgi/directory/netboot.sock
processes = 1
threads = 2
master = false
enable-threads = true

virtualenv = /path/to/wsgi/directory/venv
chdir = /path/to/wsgi/directory/
wsgi-file = server.wsgi
callable = app
```

My nginx config looks something like the following:

```
server {
    server_name your.domain.here.com;
    listen 443 ssl;
    server_tokens off;

    gzip on;
    gzip_types text/html text/css text/plain application/javascript application/xml application/json;
    gzip_min_length 1000;

    ssl_certificate /path/to/letsencrypt/fullchain.pem;
    ssl_certificate_key /path/to/letsencrypt/privkey.pem;

    location / {
        include uwsgi_params;
        uwsgi_pass unix:/path/to/wsgi/directory/netboot.sock;
    }

    location ^~ /static/ {
        include  /etc/nginx/mime.types;
        root /path/to/wsgi/directory/venv/lib/python3.6/site-packages/netboot/web/;
    }
}
```

With the above scripts, you should be able to visit `https://your.domain.here.com/`. The running uWSGI server will also manage your cabinets for you so there is no need to also use `netdimm_ensure` or any other scripts.

## Developing

The tools here are fully typed, and should be kept that way. To verify type hints, run the following:

```
mypy .
```

The tools are also lint clean (save for line length lints which are useless drivel). To verify lint, run the following:

```
flake8 .
```

## Including This Package

By design, much of this code can be used as a library by other python code, and as it is Public Domain, it can be included wherever. I would prefer that you attribute me when possible but it is not necessary. The pieces of this repo which are appropriate for external consumption have been packaged into the PyPI projects "netdimmutils" and "naomiutils". Alternatively, you can check out this repo and then run `pip install .` in the root of the checkout. The "netdimm", "naomi" and "naomi.settings" packages will be installed for you. Alternatively if you place the line `git+https://github.com/DragonMinded/netboot.git@trunk#egg=netboot` in your requirements file, then when you run `pip install -r requirements.txt` on your own code, the latest version of these packages will be installed for you. Note that by default, the webserver components are NOT included in this package. However, the "homebrew/settingstrojan/settingstrojan.bin" file is included along with "netdimm", "naomi" and "naomi.settings" as a compiled version of this file needs to exist for some of the code to work. The "naomi/settings/definitions/" directory and settings files are also included as part of the "naomiutils" package so that you don't have to provide your own copy of the settings definitions included in this repo.
