Compatibility with GameStar compatible network adaptors - do we want it? about ps2sdk HOT 19 CLOSED

I'll make the changes, but dont't expect anything soon.

I just bought a GameStar network adapter on ebay. I was surprised to see it has a SATA interface, nice! But a little disappointed to see this "network adapter" does not have a network port (only a yellow sticker covering up the fake port).

Really curious to see if this thing can do UDMA100 ;-).

from ps2sdk.

The problem was with POPStarter itself: http://www.psx-place.com/threads/open-ps2-loader-v0-9-3.13415/page-4

It likely appeared when belek666 "corrected" the way OPL passes the path to HDD-based targets.

Any issues related to changes in game behaviour may be due to some other change that is not related to this thread. I will check through the commits for issues with the EE core, since it may be related to the recent EE patch additions.

from ps2sdk.

There is a place for this discussion and the comment section of a github pull request isn't it.

from ps2sdk.

I selected the wrong registers, so I replaced the two commits mentioned earlier. They should be offset 0x00 (SPD_R_REV) and 0x04 (SPD_R_REV_3).
New commits are 48a5a34 and aa1e223.

from ps2sdk.

The more I think about it, I think I can see a reason for the DEV9 DMA callbacks. If DMA is enabled for both SMAP and ATA simultaneously, the SPEED may be unable to correctly determine where the incoming data is supposed to go. That could explain why my custom HDLDump servers in 2011 had reliability issues with the PlayStation 2 hanging, with the HDD activity indicator left on.

A long time ago, our homebrew SMAP (PS2ETH) and ATAD modules did not have support for the callbacks, and so DMA was enabled/disabled in the device driver modules before the DEV9 DMA function is used. Incoming frames were handled from within the interrupt context of the SMAP driver too, so it would have been possible to interrupt an ongoing transfer from ATA. This situation would fit the triggering conditions for such a problem if it exists...

Right now, I think I'm starting to lean towards the alternative ATAD module (i.e. atad-legacy.irx) for software to optionally use.

from ps2sdk.

GameStar
I live in europe, and somehow we have plenty of original network adapters here... I'm not sure but I think they where sold here, so I don't know why wikipedia mentions something else. I searched but cannot find any sources of them being sold here (or not). The point is, they are available, and cheap. If they are not available in another country, then they are available on for instance ebay.
IMO the only adapter supported should be the original. Clone adapters should be compatible with the original adapter, or there will be a risk of not working. It's not the software that needs to be compatible with an unknown number of unknown clones. This is also impossible for developers to test.

That being said, I realize there is a group of users (of an unknown size) that uses this adapter. There are even still clone adapters being sold on for instance ebay.
If they are compatible, then there is no problem, if they are not then i think the name of the irx file should represent the difference, like atad-legacy.irx / atad-clone.irx / atad-gamestar.irx.
Looking at the code the support is already compile-time, not run-time. So this is already a different irx file, just with the same name.

We should be happy with developers still interested in developing for the ps2 network adapter. We can't expect them to test and support clone adapters. I think in this case it would be best to split support, becouse it seems to be difficult to support them both at the same time, and the runtime is already incompatible. That way developers with an original adapter can develop and test on their hardware, and developers with clones can support their clone.

DEV9 DMA
Some time ago I made a new dev9_dma driver. It's made for (and tested in) ps2linux, but I always hoped it's design could be used for ps2sdk also. It's completely non-blocking and works for both ATAD and SMAP. The existing dev9 driver can be modified to use dev9_dma, so existing applications can still use the blocking dev9 driver call for compatibility. Would there be interest in porting the atad and smap driver to make use of this driver? Advantages are less IOP CPU load, and more speed.

Example SMAP driver using dev9_dma driver:
https://github.com/rickgaiser/linux-firmware-ps2/blob/master/smaprpc/src/smap_dma.c
Example ATAD driver using dev9_dma driver:
https://github.com/rickgaiser/linux-firmware-ps2/blob/master/pata_ps2/src/pata_ps2_ata.c

from ps2sdk.

Thanks for your input.
IMO, we live in developed countries, so shipping is somewhat affordable to us in general. It's just recently that I was thinking... and I felt sorry for those who somehow got stuck with a bad adaptor that was branded "SONY". :(
I guess, I didn't want to be responsible for wasting their money. But on the other hand, I really dislike the idea that we're being confined by the badly-designed hardware.

About your DEV9 driver, the idea is good for PS2Linux (since everything runs from interrupts), but the (re-)introduction of the semaphore can result in a performance dip for other software that run on the IOP. Originally, our homebrew DEV9 driver waited for DMA completion with semaphore (unlike the official module), but I remember that a performance test with/without the semaphore had a difference of 0.6MB/s.
In OPL, the semaphore results in some visible overhead (stuttering in audio) in some games.

But if it's included, I hope that we'll still be able to understand how the original driver was designed to work.

from ps2sdk.

Less semaphores or other task switching calls (like WaitEventFlag) is good for performance. But I don't think we need a semaphore at such a low level at all. Looking at the code I see this (correct me if I'm wrong):
level 1: ata_io_finish calls ata_dma_complete and WaitEventFlag
level 2: ata_dma_complete calls dev9DmaTransfer and WaitEventFlag in a loop untill all blocks are received
level 3: dev9DmaTransfer starts the transfer, and then enters a busy waiting loop

I think ata_dma_complete can be modified to start dma and interrupts and then wait for completion only once. The DMA completion interrupts can then be used to start the next DMA transfer (instead of the loop in ata_dma_complete). Untill the entire request is completed, then ata_dma_complete can be woken up once. This should couse less task switching, and also no more busy waiting, freeing the IOP to do other things.
Performace can be further improved by reading larger blocks (readahead), like for instance reading 4K blocks instead of 512byte would couse 8 times less sleeping and waiting. But I don't know what block sizes are used now.

I would like to test thise theories, but I have no way of comparing the changes. Is there something I can use to test performance?

from ps2sdk.

If the wait condition has been satisfied, WaitEventFlag will not cause a context switch.

What about other drivers like SMAP? For such a driver, we have to transfer data to a buffer first before marking its descriptor as valid and ready for transmission.

If you replace or remove the semaphore, then the way that threads are scheduled can change. We have been using the wrong type of semaphore in a long time (should have been set to THPRI), but that was corrected.

Since the needs for your modules are fairly different from the normal PS2SDK modules, why not just commit it as its own module?
I know that in the traditional PS2Linux kit, DMA was done by DMARelay.irx, which only relayed DMA transfers to/from the EE. Do you not perform initialization of DEV9 on the EE anymore? If initialization is no longer done on the EE, does this mean that there is no longer PCMCIA support?

The block size of PFS is 1KB, as in Linux. Zone size (Cluster) is typically 8KB.

from ps2sdk.

from ps2sdk.

I am pretty sure that there are no savings from using a semaphore, due to the frequency of frames entering and exiting the system via DEV9 DMA. My concern is more for SMAP, not ATA, since SMAP will make considerably shorter transfers that are way greater in numbers.
Yes, I did no tests on this because that was around 2012, when I had not cloned the thread monitor. But sometime ago, I did a series of experiments on why our LWIP stacks (v1.4.1 then) were considerably slower than than the custom one from SMS. The TCP thread had nearly 100% CPU utilization and throughout was much lower (1.2MB/s vs something like 1.8MB/s). The cause turned out to be the use of the kernel message-box functions. Unlike SMSTCPIP, which used its own custom message-passing functions. Getting SMSTCPIP to use SendMbx/ReceiveMbx resulted in equally sluggish performances.

The Mbx functions have a similar design to Semaphore and Event Flag functions. They disable interrupts and perform a number of sanity checks before doing what they were designed to do. Therefore, it is probably the lack of processing power by the IOP that is the real bottleneck.
In LWIP v2.0.0, this is no longer really a problem because they got rid of the need to pass messages.

Yes, not reliquishing control of the IOP will mean that the TCP thread is a resource hog, but it does not totally prevent execution of the other device threads if we schedule things correctly with priority. Actions that take place from within interrupts (i.e. SIFCMD, communicating with CD/DVD) will be also exempt from CPU hogging.

I see, so PS2Linux is now deviating from its original design (by SCEI). I agree, that is good.

from ps2sdk.

Why don't we merge the code for DEV9, but as a compile-time option? We can generate two versions of dev9.irx:
ps2dev9.irx (normal version)
dev9-advance.irx (advance version with your custom functionality)

You will then always get the latest updates for dev9, but yet the standard functions will remain compliant with their specifications.

from ps2sdk.

DEV9 DMA

You will then always get the latest updates for dev9

I'm already getting the latest updates becouse I'm loading both. The only problem with this is that dev9DmaTransfer cannot be used.

I think I can live with a compile-time option :-). But preferably have both options with the same interface:

default option:

• use the existing dev9DmaTransfer (fast)
• emulate the async interface on top of dev9DmaTransfer (slow, but compatible)

advance/async option:

• emulate dev9DmaTransfer on top of async version (0.6MB/s slower, but compatible)
• added async version (fast)

GameStar
How do you feel about splitting the device specific part from atad.irx into a separate modules? One for the original network adapter, and one for gamestar, and possibly others for other clones?

from ps2sdk.

Yeah, they can have the same interface. Didn't you already have something that already works?
I know it was sometime ago when you showed it to me, but perhaps it can be advanced and merged. Would you be doing this, or would you prefer I do it?

Regarding the GameStar: I'm leaning towards making a special version for compatible adaptors of 2015, which can be optionally used instead of ATAD (although it will open up any possible risk of using the older design).
It's not meant to be a GameStar-specific thing although the test subject was a 2015-edition of GameStar, so I would like to name the build atad-legacy... although that can be discussed. I'm still in a discussion with wisi about this, in case he has a better suggestion.

Due to how there can be an infinite number of clones, I think I would like to just support the more popular ones that do work quite well. Personally, I do own one of these compatible adaptors, but it was already incompatible in 2013. It's incompatible with HDLoader v0.8c, but compatible with "Free HDLoader". Disabling 48-bit LBA was a workaround for it, but the adaptor was still so bad that it couldn't be used to write sectors.... :(

from ps2sdk.

Yeah, there is no rush for this project. Thanks for your input earlier as well.

Also, all the best to you with your experiments! And also to getting a good edition of the GameStar.

Yeah, the compatibles/clones have SATA versions. My own compatible is a SATA network adaptor, which contains a FPGA (for the SPEED logic) and a IDE to SATA bridge (JM20330) device.

No known clone/compatible has a functional SMAP interface. I am personally not surprised, since they would need to somehow get a valid MAC address for it. Or cut corners (i.e. issue arbitrary MAC addresses), which could result in potential MAC address conflicts if the console connects to the Internet.

from ps2sdk.

@sp193 more reports on the Network Adapter issue.

It seems that the new changes made also affect an official $ony Network Adaptor that may have a SATA modification.

This reply here kind of sums it up: http://www.ps2-home.com/forum/viewtopic.php?p=19720#p19720 -- the member there lists almost all the possible setups with Network Adaptors that have an issue. Keep in mind, they all worked fine with older OPL versions prior to the changes you made.

So far, some reports are: Cloned/Fake Adaptors fail to list or run PS2 games using an internal HDD.
But then there are some with an official $ony Network Adaptor can list/play PS2 games, but can't launch POPSTARTER ELFs/VCDs via the APPS page (conf_apps.cfg). Or if they have an official Network Adaptor but has a SATA modification.

So far we have three threads at ps2-home that talk about almost the same subject: (If you have time, please read them thoroughly--I only added a quick summary below--but I'm sure I may have missed a point or two.

Thread 1: http://www.ps2-home.com/forum/viewtopic.php?f=30&t=3450 <-- this link was the primary one that we started to provide him Beta Tests -- but another member jumped in around Page 6 stating that he has a Gamestar Network Adaptor but he doesn't have any problem. And after making him test something, he then was able to report an issue: http://www.ps2-home.com/forum/viewtopic.php?p=19683#p19683

Thread 2: http://www.ps2-home.com/forum/viewtopic.php?f=19&t=3594 <-- this one is the issue reported about not being able to load POPSTARTER ELFs/VCDs from internal HDD using an official Sony Network Adaptor using a SATA modification. But it works fine with older OPL versions prior to the changes you made.

Later in that thread, I compiled the latest ifcaro repo [Open PS2 Loader 0.9.3-1008-Beta-751e52f] for him to test using the APPS page and also using conf_apps.cfg file to list his ELFs. And it still failed.

I know you are not in favor of loading POPSTARTER ELFs on OPL, but in this case, it shouldn't fail using even the ifcaro APPS page. So I figure it must be the new changes made to the internal HDD / Network Adaptor and etc.

Thread 3: http://www.ps2-home.com/forum/viewtopic.php?f=13&t=3603 <-- user reports an issue with one particular game -- also using an official Sony Network Adaptor with a SATA modification -- though we can still perhaps rule out that his rip isn't 100% good (i.e. MD5 checksum).

from ps2sdk.

@sp193 , you know, I also thought that it might have been belek666 code also. I think I did mention it before, that I wasn't sure if it was your code or his code, because both code changes were done almost the same days.

Whoa! I wasn't expecting that! Thanks for the link from psx-place.

I gotta tell you man, how far will this OPL/POPSTARTER drama go? I know that you and I already buried the hatchet with this. At the very least we've been cordially with each other after that issue. You and I still talk about OPL code and stuff.

I think you and I should make a formal announcement that you and I simply buried the hatchet. I think you and I need to do something about this before it even gets worse. It's really petty stuff. And we all know that we will never add any illegal code to OPL in order to launch the VCDs straight from it. Simply loading the ELFs shouldn't really be a problem, because that is what also the APPS page does.

Let me know if you have any ideas how we should handle this. I am already at that point that enough is enough with this petty drama. Just look at how they are now twisting this that my extra page is causing a stoppage to @rickgaiser pull requests. The last thing I want to do is discourage any new developer. I was simply reporting a symptom to him in that ticket: ps2homebrew/Open-PS2-Loader#59 (comment) -- and he was gracious enough to help.

from ps2sdk.

from ps2sdk.

To end this, I think I will follow through with wisi's suggestion... which is to just go for compatibility. The actual workaround only involves a few checks anyway. I cannot test my work, but the patches are in commits ea60bff and 6d2cb98.

Thank you all, who have participated in this discussion.

I'll close this issue. If anyone has a better way, suggestions, comments, please feel free to comment here and I will re-open this issue.

from ps2sdk.