mega65 / mega65-tools Goto Github PK
View Code? Open in Web Editor NEWTools and Utilities for the MEGA65 Retro Computers
License: GNU General Public License v3.0
Tools and Utilities for the MEGA65 Retro Computers
License: GNU General Public License v3.0
Stemmed from discussions with Tayger:
It'd be nice for commands like 'get' and 'put' to be aware of any absolute or relative path information in their arguments.
E.g. put rel/path/to/file1.prg /abs/path/to/FILE1.PRG
Think Paul was already underway with getting long-filename support in. I've also had a bite into this in the m65dbg (which contained an older copy of mega65_ftp that I tweaked). Will try to pool those efforts together on this card.
As tools like mega65_ftp and bit2core make their way to more users, it'd be good for these tools to report accurate version-information, based on the git-hash of the commit used for the release (to assist developers in tracking issues down to specific versions of the tools)
We need support for d64 and d81 uploading to the m65. This has become a blocking problem for our further development.
Some tools within this repo might've reached their used-by date and we can cull them (such as that diskmenu_c000, which has been superceded by the freeze menu's disk selector tool).
On the other side of the coin, there are probably some tools that need to be added into the TOOLS build list, such as:
See the TOOLS var in the Makefile here:
Line 63 in 095b3e1
Reported by jesperrula on M65Connect
When setting the m65 to COM port 10 or higher, m65connect will return a "file not found" error when issuing any command.
-> After testing it the problem seems to be caused by m65
Some of the latest merged has broken the repos commit history, some are merged into a squashed commit other might be lost. We plan to rebuild the history to ensure nothing has been lost.
Presently, if a long-filename is put onto the sd-card, some attempt is made to convert it into 8.3 dos format, but its a little unfriendly/unhelpful.
From my vague memory, think it was something like this:
From: actionbiker.d81
To: actionbi.ker
I'd rather it at least attempted to preserve the original file-extension (so that the m65 could still locate uploaded .d81 easily).
So perhaps logic similar to what windows does would be nicer:
From: actionbiker.d81
To: action~1.d81
(Or action2.d81 if action1.d81 is taken already)
@bubbob42 reported some problems compiling the monitor_save tool for mac here:
gcc -Wall -g -std=gnu99 -I/opt/local/include -L/opt/local/lib -I/usr/local/include/libusb-1.0 -L/usr/local/lib -mno-sse3 -march=x86-64 -o bin/monitor_save src/tools/monitor_save.c
src/tools/monitor_save.c:347:25: error: use of undeclared identifier 'B2000000'
if (cfsetospeed(&t, B2000000)) perror("Failed to set output baud rate");
^
src/tools/monitor_save.c:348:25: error: use of undeclared identifier 'B2000000'
if (cfsetispeed(&t, B2000000)) perror("Failed to set input baud rate");
^
src/tools/monitor_save.c:350:25: error: use of undeclared identifier 'B4000000'
if (cfsetospeed(&t, B4000000)) perror("Failed to set output baud rate");
^
src/tools/monitor_save.c:351:25: error: use of undeclared identifier 'B4000000'
if (cfsetispeed(&t, B4000000)) perror("Failed to set input baud rate");
^
4 errors generated.
make: *** [bin/monitor_save] Error 1
Seems like Paul feels this is a tool we should keep and make it use the m65common framework, so as to avoid such errors that were due to this old tool not using Paul's workaround to get higher serial-bps on Mac os-x, a workaround that already exists within the m65common library.
Presently, 'put' only works for single files.
Will need to add some recursive logic in order for it to recurse into subdirectories and upload each file within.
It would be nice to see the MEGA65 Tool up to date:
Taking a look at "megaflash.c" - reflash_slot():
mega65-tools/src/utilities/megaflash.c
Line 526 in 9fa7412
It appears that the code here still thinks it is dealing with 4MB-sized slots.
Paul suggests that it needs to check the FPGA version tag in the bitstream.
For more detailed thoughts on the convo that led to this card:
https://discord.com/channels/719326990221574164/791383472853614593/802163436762300456
Presently, if you try to put a file that doesn't exist locally, it creates a file on your sd-card that is zero bytes long and then fails.
It'd be nicer to just bail out early if the file doesn't exist locally and not have this un-needed empty file on the sd-card
Seems like there's a readline package available for mingw:
So perhaps we can switch back to using this rather than the present windows-specific workaround for the prompt.
I observed that M65Connect's bit to core conversion behaviour is 'slightly' different to that in bit2core.
Unsure if this is problematic and could lead to issues at this stage. Perhaps Paul has some memory of if the skipping of the 120-bytes was necessary or not. Or perhaps some inspection of how megaflash.c handles these things might shed some light.
The current code-base outputs a lot of compiler warnings.
It'd be great to clean these all up at some stage, so that in future when we make our additions, our eyes can just narrow-in on the warnings and errors those new changes introduce (rather than wading through a tonne of existing warnings and errors).
At some stage, it may be worth enforcing the "warnings as errors" setting in the compiler on the travis-side, to ensure the committer cleans these things up before merging into master.
If we got some gnu ftp tool, used 90% of it, and just ripped out the lower-level stuff that did the network comms to read and write files/dirs, and replaced it with our own code that did the same for our hardware, yeah, just maybe it'd work :)
This card can investigate that possibility.
The current CI version of m65/m65.exe very regular hangs in the middle of the transfer of block from the mounted d81 image. This is especially my observation when using the virtual mounted image to boot GEOS M65.
Got further clarification of the issue from Tayger in today's Discord thread:
The basic gist of the problem is that the -T
argument no longer seems to successfully provide a carriage return at the end of the text that it remotely types, and this was impacting M65Connect's mechanism to upload new ROMs, which after uploading the ROM, would try to remotely type a SYS58552
+enter in order to jump from c64-mode back to mega65-mode.
I'll preserve the details of the old discord thread below, though some of it may be a slight red-herring:
In discord, Tayger mentioned he experienced a problem using m65 to upload a new bitstream and latest 920181.BIN
ROM.
With the latest m65 I have loaded the latest bitstream (R2) to do some tests... Then I have loaded ROM 920181.BIN that then keeps hanging in C64 mode that way:
When I press RETURN it switches to MEGA65 mode
I assume this is m65 related
Specifics of the commands he typed are here:
There are two commands leading to the hanging in C64 mode (-4) while loading ROM. I don't know if this a m65.osx problem only.
send Bitstream:
"/Users/.../Documents/GitHub/m65connect/Code/M65Connect.debug.app/Contents/Resources/m65.osx" -l /dev/cu.usbserial-25163305977F1 -s 2000000 -b "/Users/.../Downloads/mega65r2-4.bit"send ROM:
"/Users/.../Documents/GitHub/m65connect/Code/M65Connect.debug.app/Contents/Resources/m65.osx" -l /dev/cu.usbserial-25163305977F1 -s 2000000 -F -4
Injecting a BASIC V10 prg with m65 tool does not get correctly injected.
The program get transfered and it correctly starts, but editing the program crashes the mega65.
This issue was not seen before because an assembly program mostly won't be edited after executing.
See more details on discord:
https://discord.com/channels/719326990221574164/795760112425369610/809155791718187060
Use following steps to reproduce:
Presently, the repo contains a wild variety of hard-tabs and soft tabs (or either 2-spaces or 4-spaces).
It would be good to enforce a common tab-style across the repo, so less need to chop-and-change settings within your editor/ide.
So with this card, hoping to look into whatever style-checker or linter we could quickly/easily drop in.
If it can be tied into our Travis-CI process to enforce this (at the very least, enforcing it on anything that tries to commit into master branch and rejecting such pushes), that'd be great too.
One suggestion from LAK132 was to use clang-format.
This card came about due to noticing that this repo housed a mingw/ sub-folder containing 1700 files that were used during mingw cross-compilation in linux.
I'd rather these mingw-related files weren't housed within the repo and would prefer them to live in an external apt repo instead.
Ideally, they should one day go into the official Ubuntu/debian repositories, but that might require more polish and refinement in our generated .deb files to meet their approval.
So for now, happy with any first-stab, rough-attempt at .deb file packages housed in our own private repo.
Now that we can connect tools like m65 and mega65_tools to xemu over tcp, it revealed that the conversation there suffers from some problems.
It is proving difficult to grasp the nature of all these problems, but I'll use this card to encapsulate my attempts to make it a degree better.
NOTE:
For best results, run these tools against my 220-uartmon-update branch of xemu, which contains various tweaks and improvements to the uart-monitor needed to improve the traffic flow between our tools and it.
with mingw64. initially fails with termios.h missing.
Presently, the CI-latest release page always shows a commit message dating back to 31st January 2021, despite new binaries being copied over to this release page (upon every successful merge/commit to master branch).
Some discussion on this recently in Discord here:
Use this card to look into improving this situation.
Ideally, the CI-latest page should should the commit message from the latest commit message to enter master, to give the user confidence they're downloading the latest & greatest binaries.
A little Discord discussion on this recently:
My thoughts for it are:
For a future defrag MYFILE.D81 command, behind the scenes, I'd just be doing a get, delete and put behind the scenes.
For the case of trying to put MYFILE.D81 where it already exists on the sd-card in a fragmented state, I've added a function into mega65_ftp.c called is_fragmented(filename) to check if an existing file is fragmented. So the logic here could be:
On upload_file()
{
If file exists on sd-card then
{
If `is_fragmented(filename)` then
delete existing file
}
copy across the new file
}
Based on this Discord conversation:
It got me thinking it'd be nice to get these mega65-tools talking to xemu over its tcp 4510 debug port.
I had made some steps in that direction with m65dbg already, so just a matter of porting those ideas across, perhaps into m65common.
As mentioned in the title, libncurses5-dev
is a required package to make src/mega65_ftp
.
I tested this on a fresh install of Ubuntu 18.02, running on Oracle's VirtualBox.
It'd be nice to have a del command to delete files off the sd-card.
Tayger mentioned he'll need this in order to do recursive uploads on the m65connect-side.
And besides, it's an expected command that should be available in any ftp-tool anyhow.
Tayger mentioned that it'd be great if mega65_ftp could handlhe double-quoted parameters wit spaces, so that commands like this would work:
put "long spaced filename.d81" FILE.D81
After loading ROM 910523.bin by m65, any loaded PRG file intended for C65-mode, stalls with this output:
m65 -l COM9 -R 910523.bin
m65 -l COM9 -r "Turbo Disk 65 v3.prg"
...
[T+0sec] Detecting C64/C65 mode status.
[T+2sec] Could not determine C64/C65/MEGA65 mode.
[T+2sec] Loading file '/.../.../Turbo Disk 65 v3.prg'
[T+3sec] CPU in C65 BASIC 10 main loop.
[T+3sec] Injecting string into key buffer at $02B0 : DLo"![$0d]
[T+3sec] Waiting for breakpoint at $F664 to trigger.
NOTE: This issue only occurs in C65-mode. I.e., loading a c64 .prg via m65 -l COM9 -4r commando.prg
still works.
It looks like m65 has a problem to evaluate the MEGA65 mode after loading a ROM file.
Loading a PRG file with RESET option works but that of course removes the ROM file.
But loading a PRG after 911001.bin works (also several times).
Latest r3 bitstreams exceed 4MB, but the bit2core tool presently only permits them to be 4MB max.
I want to add an extra argument to specify the intended board revision for the outputted core.
So if "r2" is specified, it enforces a max bitstream size of 4MB.
If "r3" is specified, it enforces a max bitstream size of 8MB.
It'd be nice if our Jenkins builds could also cross-compile for Mac OSX too, as we could then get some early feedback on whether any commits break builds for mac osx.
A quick google found this SO thread which had a few suggestions which could be considered:
For upcoming commands I've added to mega65_ftp, one such as "secdump" command will trigger many calls to read_sector() to retrieve potentially several megabytes worth of data.
I observed the speed of transfer was good up until 2MB has been downloaded. After this point, it slowed down.
A while later, at around 3MB I think, I got a seg-fault.
The slowdown after 2MB seems to relate to the sector cache being this size (4096 sectors x 512 bytes), so it appears I've exhausted the cache.
I found this hard to debug on my end, maybe it made a difference which gcc I compiled it under. My linux netbook has two gcc's now. Think I got the seg-fault on the newer gcc (which I couldn't debug well with my older gdb). When I build m65ftp with my older gcc, I can debug with gdb, but I don't get the segfault.
Anyways, for now, I've applied a workaround in my branch, inside read_sector(), if it notices the cache has been filled, it will just empty it, hence preserving the higher speed transfer through the entire course of my secdump command.
Based on this discord discussion:
https://discord.com/channels/719326990221574164/805492126128930846/806653520589357066
As for the error message you got when doing the put:
-c "put /Users/.../Downloads/Zauberschloss2.prg"
Zauberschloss2.prg does not yet exist on the file system -- searching for empty directory slot to create it in.
dir_cluster = $2, dir_sector = $6128
Uploaded 18432 bytes.ERROR: Short read: Only got 368 bytes
Uploaded 19312 bytes in 1 seconds (18.9KB/sec)
Yep, I've seen this error before on my side, even on a successful put.
It's something I haven't debugged yet, though interestingly, if I do a get of the uploaded file immediately after such a put, and then diff the two files, they are identical.
It'd be great to have some logic for .d81 uploads to find a location on the sd-card with sufficient consecutive available sectors in order to prevent defragmentation of the file. Presently no such logic exists in the tool.
The current source is a mix of tabs and spaces, it'd be nice to clean this up to be consistent. Not sure what is the preferred style,
Trying to infer from what I've seen, it seems like 4-spaces per tab seems to be the preference. Perhaps they should always be genuine spaces then and no tab character?
There's a bit of rigmarole involved for Windows users to get bitstreams uploaded to their devices via the '-b' argument.
So if they attempt to, and it fails, I'd like to point them to a page with information on the preparatory steps, most of which I've outlined here (but perhaps it could have a better home somewhere else too):
It would be nice if there was an option to do a softreset instead of a full core reset when sending a file to the Mega65. It could save a lot of waiting time each time we compile and run a prg.
Any other ideas for making the push of PRG's to the Mega65 faster would be very welcome too.
Was pondering this on Discord recently:
Hmm, another thing I'm slowly starting to ponder is the possibility of adding unit tests for tools like bit2core, perhaps even bdd tests for tools like mega65_ftp.
I'll admit writing tests is not a strength of mine, but maybe it's a chance to grow that skill, and I'm feeling more and more that tools like bit2core and mega65_ftp deserve them, as we'd like to minimise the possibility of tools like this screwing up people's sd-cards and whatnot :-)
Anyway, at work, we use gtest for unit tests and an in-house framework for bdd tests.But mega65-tools repo is predominantly c-code, maybe gtest isn't the best fit? (More suitable for C++?).
Anyways, I'm open to hearing other people's thoughts and experiences on unit/bdd testing c-code, and perhaps a recommendation or two?
I'll do my own googling and reading also, but happy for additional input.
Paul hinted that the bitstream header contains information that we can extract to ascertain what device the bitstream was intended for.
The hope being that we can prevent a user from inadvertantly making use of the wrong bitstream for their device, via such a check within bit2core tool.
I.e., We've recently forced the user to provide an argument specifying their device (either r2 and r3 at present, I can add 'nexys' in this card too). So in addition, we can assess if the bitstream they provide contains target-device information that matches this choice.
So far, the best source for this target-device info I've observed in the bitstreams is located at offset 0x4c as a 12-byte string, as seen in this comparison between the 3 device bitstreams (Nexys4DDR, Mega65r2 and Mega65r3):
This reveals the following strings:
With cc65.exe V2.19 - Git bc8cca9 you will get the following error when executing make
. I'm using mega65_ftp but could be a general issue.
$ make USE_LOCAL_CC65=1 bin/mega65_ftp.exe
git submodule init
git submodule update
cl65 --config src/tests/vicii.cfg -I src/mega65-libc/cc65/include -O -o src/utilities/remotesd.prg --mapfile src/utilities/remotesd.map src/utilities/remotesd.c src/mega65-libc/cc65/src/*.c src/mega65-libc/cc65/src/*.s
1 errors and 0 warnings generated.
src/mega65-libc/cc65/src/conio.c(499): Error: Conflicting function types for 'cputcxy'
make: *** [Makefile:229: src/utilities/remotesd.prg] Error 1
This is already fixed at libc, so we need to update cc65 submodule to latest (see MEGA65/mega65-libc@068898d)
Paul commented on another PR that he plans for the .core file's header to also contain a field to define which target the .cor file is intended for, so that on the M65-side (megaflash.c), this detail can be checked and core-slots can be managed accordingly.
Paul: (27 Jan) this reminds me, we should actually indicate the intended target for a .COR file in the 4KB header somewhere. Then we can modify the flash menu to check that, before trying to flash it.
In this card, I've proposed to add the M65_TARGET field in the following location within the core file's header (at offset 0x50).
In further conversations with Paul on Discord, he suggested the addition of a model-id field that will equate to the value of the $D629 register on the m65-side.
So I'll also aim to add a M65_MODEL_ID field into the .cor header (at offset 0x70):
0x0000 +====================+
| "MEGA65BITSTREAM0" | (magic string - 16 bytes long)
0x0010 +--------------------+
| CORE NAME | (32-byte string)
0x0030 +--------------------+
| CORE_VERSION | (32-byte string)
0x0050 +--------------------+
| M65_TARGET | (32-byte string)
0x0070 +--------------------+
| M65_MODEL_ID | (1-byte)
0x0071 +--------------------+
| |
: CORE PADDING :
| |
0x1000 +====================+
| |
: BITSTREAM HEADER :
: & DATA :
. .
In that Discord thread, I devised a table to map what fpga-part name is associated with each model-id. I can make use of these mappings in my implementation:
// Mega65 Target Name FPGA Part Model ID
mega65r1 7a200tfbg484 $01
mega65r2 7a100tfgg484 $02
mega65r3 7a200tfbg484 $03
megaphoner1 7a100tcsg324 $21
nexys4psram 7a100tcsg324??? $40
nexys4ddr 7a100tcsg324 $41
nexys4ddrwidget 7a100tcsg324 $42
wukonga100t 7a100tfgg676 $FD
simulation n/a $FE
This update will require changes to the following tools:
Presently, I suspect 'del' command only works on files. Tayger recently put on my radar the desire to delete directories too, which I don't think the current implementation considers well.
Will probably need to devise a recursive algorithm that dives into all sub-directories too and deletes all files within there also.
I find cc65 build output overly clutters the travis-ci build output, and I'd rather limit the build-output focus to the mega65-tools that are being built.
Perhaps cc65 could be packaged up nicely into a debian package, suitable for installation via apt-get?
This seems to have been broken for a long while, with an alternative tool called xc3sprog
being suggested to be used by Windows developers for their bitstream-uploading needs.
I've already done some initial investigation the matter, some findings in this discord thread:
To summarise some experiences so far:
Main distinction in behaviour I see right now is in the time that elapses between the "fpgajtag: Starting to send file" and "Bitstream loaded." messages
So in this card, I'll try and dig deeper into libusb to try understand what it going wrong in this attempt to write the bitstream to the device in windows.
If I'm unable to resolve the problem, perhaps I can just put a warning inside m65.exe so that it suggests using the xc3 tool for bitstream uploading instead.
17:02 1076 /Users/home/MEGA65/C65/tools cc -o romdiff romdiff.c
romdiff.c:369:3: warning: 'snprintf' will always overflow; destination buffer has size 224, but size argument is 256 [-Wbuiltin-memcpy-chk-size]
snprintf((char*)&header[32], 256, "%s", normalise(argv[1]));
^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/secure/_stdio.h:57:3: note: expanded from macro 'snprintf'
__builtin___snprintf_chk (str, len, 0, __darwin_obsz(str), VA_ARGS)
^
romdiff.c:370:3: warning: 'snprintf' will always overflow; destination buffer has size 160, but size argument is 256 [-Wbuiltin-memcpy-chk-size]
snprintf((char*)&header[32 + 64], 256, "%s", normalise(argv[2]));
^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/secure/_stdio.h:57:3: note: expanded from macro 'snprintf'
__builtin___snprintf_chk (str, len, 0, __darwin_obsz(str), VA_ARGS)
^
2 warnings generated.
the following line is missing a trailing ')':
https://github.com/MEGA65/mega65-tools/blob/master/src/tools/mega65_ftp.c#L628
Recently, we discovered a problem on converting recent .bit files to .cor, discussed in this Discord thread:
$ bin/bit2core.exe mega65r3 ~/Downloads/mega65r3_b70.bit mega65r3-b70 mega65r3-b70 mega65r3_b70.cor
Bitstream file is 6165570 bytes long.
This bitstream's FPGA part is for an unknown mega65 target: "???" (FPGA part: )"
ERROR: Provided bitstream is for a different mega65 target to the one you specified: "mega65r1|mega65r3" (FPGA part: 7a200tfbg484)
Looking into it, bit2core assumed that the FPGA-part-id field would always be located at offset 0x4C.
00000000 00 09 0f f0 0f f0 0f f0 0f f0 00 00 01 61 00 39 |.............a.9|
00000010 63 6f 6e 74 61 69 6e 65 72 3b 43 4f 4d 50 52 45 |container;COMPRE|
00000020 53 53 3d 54 52 55 45 3b 55 73 65 72 49 44 3d 30 |SS=TRUE;UserID=0|
00000030 58 46 46 46 46 46 46 46 46 3b 56 65 72 73 69 6f |XFFFFFFFF;Versio|
00000040 6e 3d 32 30 31 39 2e 32 00 62 00 0d 37 61 32 30 |n=2019.2.b..7a20|
00000050 30 74 66 62 67 34 38 34 00 63 00 0b 32 30 32 31 |0tfbg484.c..2021|
But looks like in this new bitstream, it is at offset 0x4e.
00000000 00 09 0f f0 0f f0 0f f0 0f f0 00 00 01 61 00 3b |.............a.;|
00000010 63 6f 6e 74 61 69 6e 65 72 3b 43 4f 4d 50 52 45 |container;COMPRE|
00000020 53 53 3d 54 52 55 45 3b 55 73 65 72 49 44 3d 30 |SS=TRUE;UserID=0|
00000030 58 46 46 46 46 46 46 46 46 3b 56 65 72 73 69 6f |XFFFFFFFF;Versio|
00000040 6e 3d 32 30 32 30 2e 32 2e 32 00 62 00 0d 37 61 |n=2020.2.2.b..7a|
00000050 32 30 30 74 66 62 67 34 38 34 00 63 00 0b 32 30 |200tfbg484.c..20|
I've found a description here of the xilinx bitstream file format:
Ah ok, that doc helps a lot, they use variable-length fields. I'll need to tweak the logic inside bit2core to suit this...
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.