sakura-it / sonnetamiga Goto Github PK

As @DvdBoon noticed, the m68k version of vasm has some optimisations enabled by default. This results in some (unwanted?) behaviour in the newest version of vasm.

For example in getinfo binary 2169 fd56 0000 becomes 20a9 fdb0.

The optimisations can be disabled by adding -no-opt parameter to assembler flags (which I just did), but this results in many more minor differences.

Sonnet library now holds one pagetable and one BAT setup for all programs. Real WarpOS gives the opportunity to programs to set up protected memory for themselves (using special MEMF flags).

Introduce this system also for the sonnet library for optimal compatibility.

(I've said I would never do this, but latest changes to internal builds resulted in running program without the PPC MMU. This does mean however that the 68K MMU needs to come in play. And also mmu.library).

As a temporary solution, we should support initialising graphics card memory from sonnet.library. This is supposed to be a temporary solution since KlozetPCI is not ready to use yet, so that we can avoid using pci.library.

Add support for at least Voodoo 3. I'm digging through the datasheet, I'll see what I can do.

When not having a lot of gfx memory (read: voodoo card) and playing the Quake II intro (which is rather big) PPC code and/or data residing in Sonnet memory gets overwritten if you let the intro play long enough. This does not happen on a Radeon card with 256MB.

I'm guessing that the graphics driver assumes it has all $2005 memory for itself. And that allocating memory for bitmaps does not go through the normal memory allocation functions but through P96 or CGX.

Will look into it.

This in combination with programs overwriting graphics memory results eventually in a hang of the system. (Time dependent on amount of VGA memory).

For example: Quake II does not release memory upon switching to a different map. The loaded maps will fill up the memory eventually.

The new vlink version (source snapshot as of two days ago) fails to link library with the following error:

Error 32: Target amigahunk: Unsupported relocation type R_PC (offset=0, size=16, mask=ffffffffffffffff) at LibBody+0x1ea.

Investigating.

@DvdBoon is having problems running the new version of vlink (current source as of 2015-03-03) on his AmigaOS system.

While linking the executable, proper binary is generated by vlink, but 8000 0004 is thrown upon exit. Apparently the cause is a jump to -186(a6) with a6 being 0.

No issues were noticed on a Linux build server. I'll try to reproduce this on my AmigaOS system and then report the problem to vlink author.

Now that we have C compiler producing mostly good executables for Sonnet, I'm slowly working on integrating wosdb into our build system (issue #9).

I've ran into a problem with macro while assembling PPC parts of wosdb. There's a CALLWOS macro in wosdb/warpos_lvo.i (similar to our CALLPOWERPC) that uses NARG variable to extract the number of parameters passed to the macro. However, it seems that NARG is not supported for std syntax in vasm (orignally wosdb used pasm).

This results in the following error:

error 10 in line 1 of "CALLWOS": number or identifier expected
    called from line 64 of "asmdebugger.s"
>   .ifgt   $NARG-1

Any ideas with what this NARG construct can be replaced? In worst case I can just split this macro into two separate macros but I feel that would be less elegant.

I have 2 PRPMC800-1251 boards with PMC239/F carrier cards. These boards feature 128mb RAM. Mediator would normally have the WinSize jumper open with less than 256mb RAM, but InitPPC will not run unless jumper is closed. With it closed, the boards are working, but I've noticed that ShowConfig describes them having 512mb RAM, and wonder if this mismatch of pci memory space contributes to stability issues of Apocalypse. I am also observing that pciinfo reports mine with Vendor ID $1057 but Device ID: $480b with DeviceName unknown instead of MPC107 and doesn't list a MemSpace address. I've checked the latest vendor.txt from Elbox and it does not include this device ID. GetInfo results and Warp working, and not sure significance of these observations. I had jumpered the boards 11-12 as closed because I'd read to somewhere, but cannot recall what the jumper setting does. Hoping for clarification from someone with more expertise, or possibly share something that may improve the compatibility.

Hello, Sakura,

I do not have any other way to contact you. I am wondering if you know whether a Bigfoot Killer NIC K1 will work with a 1200 mediator board?

Regard.

While investigating optimisations issues, I discovered it is impossible to use IRA on binaries generated with the new version of vasm and vlink (with the memory attribute set to $1005).

Running IRA on such binary results in:
Hunk...:00001005 NOT SUPPORTED.

I'll report this issue to author of IRA and ask him to add support for such hunks.

If the user tries to run a binary referencing powerpc.library, he'll just get the error message saying it was not found. The sonnetpatch utility should be able to change these references to use sonnet.library instead.

Sometimes, programs like Quake or WipeOut show lag when started, quit, and started again.

The initial start does not show any lag in fps. After the second start the fps drops sharply. This is not 100% reproducable on my system. Sometimes it takes a couple of restarts of the game to get the effect.

After completing the first level the sound is sped up. This is probably due to some persisting/wrong signals.

Apparently AmigaOS 3 executable loader is not very strict about enforcing the format of binaries. I have several examples of programs that work just fine, but their hunk headers are broken in various ways. Most common offsense is wrong length of hunk specified in the hunk header. Sonnetpatch hunk parser needs to be extended not to fail on such programs.

The source code of this project is provided here on GitHub, but it is not licensed. We have to decide under what license will the program be distributed.

There are two common options in the open source world:

• BSD license (typically 2-clause version is used for new projects): http://opensource.org/licenses/BSD-2-Clause
• GNU General Public License (newest version is 3): http://www.gnu.org/copyleft/gpl.html

We should carefully think about which license will be better, knowing the peculiarities of Amiga community.

All the $VER strings should follow the format described in Amiga Developer Docs.

http://amigadev.elowar.com/read/ADCD_2.1/AmigaMail_Vol2_guide/node0011.html

This is certainly the right format for AmigaOS 3.1. Later AmigaOS versions added ability to use 4 digit year and a comment in version string. But I think we should stick to OS 3.1 format, since that's the lowest version of AmigaOS where our implementation of powerpc.library can function correctly.

In powerpc.library, the signals between the PPC task and its 68K mirror task are shared. This is not the case yet within sonnet.library. This leads to dedicated PPC sound threads as in Earth2140, FreeSpace and SCUMMvm going into infinite waiting state as they are not signaled properly back by their 68K mirror task.

For example: SCUMMvm PPC sound thread issues a SendIO() to AHI using Run68K. It then checks immediately with CheckIO() using Run68K if AHI is done. When not done, the PPC thread goes into WaitPPC() never to be woken up by the 68K mirror task when AHI is actually done.

vasmppc_std/vlink cannot handle the memory attributes assigned to a code section.

For example:

.section "ppccode","acrx",0x1005

does not work.

Looking at the output of WarpRace regarding context switches on the 604e of the Cyberstorm and the Sonnet there is an order of a magnitude of difference (1.500ms versus 15.000ms). This is probably linked to WarpRace executing 68k code within sonnet memory and/or reading data with the 68k from sonnet memory but I am not sure. Will have to look into it.

During stress-testing of the code, the following issue was detected:

Running different, moderate complex, WarpOS programs in concession crashes the PPC.

For example

1. Start sonnet.library -> any number of Getinfo mixed with any number of CyberPI OR CyberMand
   -> Works

2. Start sonnet.library -> mix CyberPI AND CyberMand
   -> PPC crashes during the start of the second program (being CyberPi after Cybermand or vice versa)
   -->In some occasions, CyberPI runs but calculates PI wrongly.

This can be a memory allocation issue, register transfer issue, program exit code issue or a cache issue. As there are no debuggers for the sonnet available (yet) it takes time to pin-point the exact cause.

During stress-testing of the current code the following issue was revealed:

PPC CPU wanders into never-never-land when a delay is omitted during initialization. It looks like an MMU problem but as there are no debug tools (yet) for the sonnet it is hard to pin-point the exact cause.

For example:

1. compile code
2. start sonnet.library
3. start getinfo

-> works

1. reset
2. start sonnet.library
3. start getinfo

-> does NOT work

For now the delay inserted into the initialization code works around the problem, but this is an unwanted situation

I have started to remake the library into C for easy of maintenance and as a learning project.

Current concerns:

1. Uses Elbox NDK
2. Library copies itself to other part in RAM (PC relative/Reloc in C?)
3. PPC functions are currently in some PowerOpen ABI which does not seem the same to the VBCC output (StormASM uses r13 as a user stack pointer and vbcc uses r1 for everything.
4. The PPC does not interrupt tasks that are currently inside list functions (due to missing Enable()/Disable() functions in WarpOS). Not sure how to implement this in C, Now a check is done on memory range. But functions within C can be rearranged by the linker.

It may be an idea to flush the 68K caches properly when doing a context switch from 68K to PPC. There are actually only a few points in the code where a flush needs to be: In the Run68K code on the 68K side and when sending a packet to the PPC.

Not sure how this will influence overall speed of the context switch. Maybe make it an option. Maybe use the mmu.library in the sonnet library to set things directly instead of a config file?

Use openpci.library so it works on more hardware (enhancement)

At the moment iFusion is not working. It depends on some advanced features of WarpOS. If we ever get iFusion running we can effectively say that we are done regarding WarpOS compatibility :-) This is why I want to add a separate issue to it.

Debugging sessions have not taken me far yet. The first hurdle was exception handlers. These have been implemented and the large_context ones work (they are used by wosdb). iFusion however uses small_context exception handlers and are not fully tested yet.

At startup, iFusion sets up a number of exception handlers and jumps to the program/trap/privilege exception handler. Here it hangs due to loading a wrong value into SDR1 (needed for MMU setup).

Recently i discovered that this is due to alignments not supported by AllocVecPPC(). iFusion allocates 0x400000 bytes with alignment 0x100000 for the page table. It gets a non-aligned (well, actually 0x20 aligned) block in return. This is invalid for SDR1.

Hope to add alignment support for AllocVecPPC() soon. I also foresee problems/conflicts with the default MMU setup of the library. but first things first.

Hi,

since the Bigfoot Killer NICs K1/M1 are now very hard to come by, and since I happen to have a Killer NIC 2100 with 400MHz "NPU" lying around, might it be possible to:

a) Run this card in an PCI to PCI-E adapter like this one https://www.startech.com/Cards-Adapters/Slot-Extension/PCI-to-PCI-Express-Adapter-Card~PCI1PEX1 (I can't imagine why this shouldn't work, but perhaps someone here knows better)
b) In the above configuration, support this card via the sonnet.library.

Thanks in advance,
Torsten

The scheduler sometimes 'forgets' tasks. The task gets placed into a wait state and is never activated again. This is, for example, seen in WarpSNES (Mostly the sound task, so sound stops playing).

There is no crash. The PPC can still accept new tasks.

Problems

• Currently sonnet.library is relying on a modified version of Elbox's pci.library.
• Currently sonnet.library requires an access to graphics card memory, which in some cases require prior card-specific setup (for example initialisation of memory controller on Voodoo 3).
• Elbox does provide development kit only under restrictive NDA.
  • We don't have an access to that SDK.
  • Such restrictive NDA is incompatible with open source nature of this project.

Proposed solution

• Write a new, open source implementation of PCI library - codename KlozetPCI.
• Use NetBSD Mediator drivers as a basis.
• Provide bus_space(4)-like API.
• Optionally add a compatibility layer that would allow to use existing drivers (OpenPCI or Elbox pci.library API reimplementation).

Next to the speed up of sound (see #37) there is also content missing in Shogo. Voices are missing during the start dialogue (only self is heard). Portraits with the spoken text are missing. Intro text during start new game is missing (press esc to skip).

These items work as intended with ReWarp.

The following does work correctly with vasm from Mar 12 2015 but fails with vasm built today:

.macro CALLWOS 
        .ifnb \2
                mr r3,\2
        .endif
        lwz r0,\1+2(r3)
        mtlr r0
        blrl
.endm

Throws the following error:

error 10 in line 2 of "callwos": number or identifier expected
    called from line 64 of "asmdebugger.s"
>       mr r3,\2

Using explicitely named arguments \foo instead of \number works correctly.

Investigate.

Currently, C executables linked with Sonnet-modified WarpOS target (sonnet.o instead of warpup.o) still contain two hunks without 0x2005 extended memory attribute.

This can be observed for example, in wosdb. Even though I added a bunch of -hunkattr parameters to vlink.

Investigating, I suspect x.o can be blamed here.

Note hunk 6, 7 below:

$ sonnetrun/a.out wosdb/wosdb 
HUNK_HEADER
    Hunk table size: 9
    First hunk to load: 0
    Last hunk to load: 8
Unaligned read at 15f60!
Unaligned read at 15f60!
HUNK_CODE (0x3e9) hunk number 0 at offset 0x54
    Size: 156 Amiga longwords (624 bytes)
    Flags: Extended mem attributes
        Extended memory attribute: 0x2005
    Relocation: HUNK_DRELOC32 (0x3f7)
HUNK_CODE (0x3e9) hunk number 1 at offset 0x2cc
    Size: 17616 Amiga longwords (70464 bytes)
    Flags: Extended mem attributes
        Extended memory attribute: 0x2005
    Relocation: HUNK_RELOC32 (0x3ec)
HUNK_CODE (0x3e9) hunk number 2 at offset 0x1162c
    Size: 7 Amiga longwords (28 bytes)
    Flags: Extended mem attributes
        Extended memory attribute: 0x2005
    Relocation: HUNK_DRELOC32 (0x3f7)
HUNK_DATA (0x3ea) hunk number 3 at offset 0x1166c
    Size: 829 Amiga longwords (3316 bytes)
    Flags: Extended mem attributes
        Extended memory attribute: 0x2005
    Relocation: HUNK_DRELOC32 (0x3f7)
HUNK_DATA (0x3ea) hunk number 4 at offset 0x12378
    Size: 1349 Amiga longwords (5396 bytes)
    Flags: Extended mem attributes
        Extended memory attribute: 0x2005
    Relocation: HUNK_DRELOC32 (0x3f7)
HUNK_DATA (0x3ea) hunk number 5 at offset 0x13c98
    Size: 116 Amiga longwords (464 bytes)
    Flags: Extended mem attributes
        Extended memory attribute: 0x2005
    Relocation: HUNK_DRELOC32 (0x3f7)
HUNK_DATA (0x3ea) hunk number 6 at offset 0x13f34
    Size: 4 Amiga longwords (16 bytes)
    Flags: None (implied MEMF_PUBLIC)
    Relocation: HUNK_DRELOC32 (0x3f7)
HUNK_DATA (0x3ea) hunk number 7 at offset 0x13fdc
    Size: 6 Amiga longwords (24 bytes)
    Flags: None (implied MEMF_PUBLIC)
    Relocation: HUNK_DRELOC32 (0x3f7)
HUNK_BSS (0x3eb) hunk number 8 at offset 0x14010
    Size: 1996 Amiga longwords (7984 bytes)
    Flags: Extended mem attributes
        Extended memory attribute: 0x2005
    Relocation: HUNK_DRELOC32 (0x3f7)

A major slowdown is noticed when InitPPC is part of the startup-sequence. This has been pinpointed to DefIcons in WBStartup. Removing/Renaming ENVARC:deficons.prefs solves this. It appears that the reading of files of DefIcons conflicts with the LoadSeg patch of the library.

Frank Wille once wrote a WarpOS debugger called wosdb. Apparently it does everything with WarpOS API. Not sure if it would be useful for debugging sonnet.library itself, but definitely would help fixing applications running on top of it.

Investigate and make necessary adjustments to make it compatible with sonnet.library.

Currently automated builds are done with a hand-crafted script. Long story short, it sucks. Implement Jenkins to control the build process, report, store history, alert via mails etc. etc.

http://jenkins-ci.org/

At the moment, the sonnet.library only supports the A3000Di mediator. This is a mediator which has the needed 3.3V line. The only other mediator which has the 3.3V line (and with a much larger userbase) is the mediator TX (when connected to an ATX PSU).

The current state is that the PPC is initialized by the sonnet library. It sets up its memory and communicates this with the 68K. The 68K however cannot properly address the sonnet memory. This is probably due to the Z2 window.

I suspect there are functions inside the pci.library to fix this. The memory of the sonnet should be initiated the same way as graphics memory (using the pci.library) and the relevant pci.library functions will probably contain MMU code.

I will investigate this further.

Need to clean-up the repository.

• remove sonnet.library
• remove special wosdb
• remove sonnetpatch
• remove InitSonnet
• remove VBCC enhancements (x.s / startup.s)
• remove leftover references to IRA (we didn't reassemble in automated builds in a long time)
  ... more?

Appears in:

ADoomPPC (when exiting)
WipeOut2097 demo (5 times at the start)
Earth 2140 Demo (2 times start, 1 time end)

Have to verify if it isn't related to my system only.

Typical WarpOS applications are linked to place data and code in Fast RAM. For it to be accessible by PPC CPU, it should be placed in memory with attribute 0x1005 (which is Sonnet memory).

How to patch existing programs? I guess we need some kind of wrapper that takes an executable, patches all PPC hunks and then starts it?

I wanted to plug InitSonnet into our build system. Basically I've done the necessary adjustments, but do we really need that DevPac system library? It makes requirements on external stuff even more complex than it already is. If possible, I'd like to keep them at minimum. What does InitSonnet really use that is there? Can we just copy^Wreimplement the necessary routines and put them in our tree?

Hi Guy's,

would be great if you think about supporting the Killer Xeno Pro and Killer 2100 PCIe cards using PCI to PCIe adapter. The cards do have a 400MHz CPU (dual?) and 128MB of DDR2 Memory on board. Would be a great and cheap upgrade!

I would be happy to help supporting the cards cause I already own one so I'm able to do tests with it!

// Thomas

@DvdBoon
Some files are duplicated in both directories:
ppcdefines.i
ppcmacros-std.i
Is it safe to move them to one directory, let's say named "common" and just add that directory to include path in both sonnetlib and tools with -I parameter to vasm? It could work this way from both Makefiles and AmigaOS script that you're using.

I guess it should be fine, but I preferred to ask before moving anything since it's your code ;).

Voodoo4/5 gets put on an 128MB boundary if it is configured after a Sonnet. It needs an 256MB boundary for it to work with the Sonnet. A quick fix is to place the Sonnet above the Voodoo4/5, but in case of an A4000D, the Amiga cannot be closed.

It is partly due to pci.library

Structures returned from Run68K() calls to, for example, OpenWindowTags() from the intuition library cannot be manipulated by the PPC if the are in Amiga Fast memory.

Either we force placement of these structures into sonnet memory by patching the Allocmem() function or we brute-force it by upping the priority of the sonnet memory. The latter slows the Amiga a lot.

Another possibility is to catch all Fast memory access through the PPC MMU. This will slow the PPC a lot.

Any other thoughts?

AmigaOS NDK 3.9 is missing some important includes. It is not possible to build the project without fixing these dependencies manually.

Investigate and provide an elegant solution.

I want to document here the programs which are working and which are not. Hopefully in time the first list will get longer and the second one will disappear :-)

The list of working/non-working software is kept in the Wiki:
https://github.com/Sakura-IT/SonnetAmiga/wiki/compatible-software

After upgrading vasm, every call to FUNCDEF macro ends with the following error.

error 68 in line 2 of "FUNCDEF": repeatedly defined symbol <FUNC_CNT>
    called from line 12 of "exec/exec_lib.i"
    included from line 3 of "sonnet.s"
>FUNC_CNT    SET    FUNC_CNT-6  * Standard offset-6 bytes each

Mailed Frank about this.

Using AHI together with a sound card results in a distorted sound. Using the Paula audio-modes from AHI does not give this distorted sound.

Current hypothesis is because there is no priority-based scheduler yet in the sonnet.library (all tasks get equal time) the sound task gets not enough cpu time (switches away too much).

Currently graphics card memory is only used to boot the PowerPC CPU. Executing PowerPC programs requires memory installed on the Sonnet board.

Sonnet Crescendo 7200 has 3 slots that accept 5V, 64-bit-wide, 168 pin, fast-page mode, and 70 ns or faster DIMMs. They are quite difficult to find these days, so an alternative method to run the code should be provided.

It would be nice if we could also use graphics card memory (in case where no Sonnet memory is installed) to run the PPC programs from it.

WarpOS keeps track of the time a task spends in running or idle. This is not implemented in sonnet.library.