anders429 / simfile Goto Github PK

License: Apache License 2.0

Rust 41.48% JetBrains MPS 58.52%

simfile's Introduction

simfile

Reading, writing, manipulating, and converting simfiles.

A simfile is a file used in a rhythm game simulator. This library provides tools for reading them, creating them, modifying them, and transcoding between them. The intention is to have a one-stop shop for working with all formats. A number of common simfile types are supported, including:

.msd
(More planned for future versions)

Usage

This library provides a Song type, which stores the data for a single simfile song. A Song can be read directly from an existing simfile of any supported type.

Example

Reading a Song from a .msd simfile is done as follows:

use simfile::Song;

let song = Song::read_msd("simfile.msd").expect("error while reading `.msd` file");

Similarly, writing to a .msd simfile is done from an existing Song as follows:

song.write_msd("simfile.msd").expect("error while writing `.msd` file");

Note that both Song::read_msd() and Song::write_msd() return Result types that must be handled appropriately.

Minimum Supported Rust Version

This crate is guaranteed to compile on stable rustc 1.58.0 and up.

License

This project is licensed under either of

Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

simfile's People

Contributors

Watchers

simfile's Issues

Skip unknown tag names

Should be possible now that Anders429/msd#24 is published. This will give way more flexibility in parsing simfiles, and actually allows a few more test simfiles to be successfully deserialized.

Misses in log code coverage

There are some misses in code coverage in the internal_log.rs file. You can see them here.

All of these misses are in the tests, and I don't think there's really any way we can remove the misses, since they're occurring when log is deactivated, which means all of the statements become no-ops and are therefore never run.

tarpaulin does provide a flag for ignoring tests, and that seems to be the best way to deal with this. After all, we care about code coverage in the actual code, not in the tests.

Add `genre` field to `Song`

This field will represent the musical genre. Should be an optional string.

Warps as negative BPMs

Warps originated in .sm files as a glitch exploit using negative BPMs. These should be supported to maintain backwards compatibility, as a lot of popular .sm files used and abused this mechanic.

However, it seems .dwi files also had the same capability. Here is a .dwi file and its counterpart .sm file, both of which use negative BPM and stop values.

Investigation needs to be done to determine if the functionality was the same in the original Dance With Intensity simulator. Additionally, other file formats need similar investigation, such as .msd with the original DDR'99 simulator.

Define `Song` struct

A canonical Song struct should be defined that allows storage of song simfile data for all song file formats.

The struct will need to support header information (such as title, artist, etc.), alongside repeated chart data (steps, timing, etc.). This format will likely resemble the SSC format the closest, but will also allow storing data specific to other file formats. The structure will be able to be mapped both to and from each of the respective file formats.

Most of the fields will be optional, since not all of the fields will be provided with each respective format.

Header fields to be included:

Chart fields:

Enable `cargo_common_metadata` clippy lint

This lint ensures that all necessary manifest information is provided. CI will ensure that this information is consistently provided.

See https://rust-lang.github.io/rust-clippy/master/#cargo_common_metadata for more info on the lint.

Add `origin` field to `Song`

Should represent the song's origin (as in, what collection it was originally from?). Should be an optional string.

Enable dependabot

Dependabot should be enabled to ensure that dependencies stay current.

Parsing should preserve character location

When parsing, line and character location should be preserved to allow for better error messages.

Code Coverage issue with `log` feature

When log is enabled, different tests are run than when it is disabled. This causes the internal_log.rs file's tests to only be half-covered, regardless of the features enabled.

It looks like tarpaulin supports a config file, which can be named .tarpaulin.toml. It appears, from the example, to allow specifying multiple configurations for coverage runs. As far as I can tell, they seem to be combined into a single report when tarpaulin is run.

Set up GitHub Actions

GitHub Actions should be set up for continuous integration. To start with, the following actions should be implemented:

tests
rustfmt
clippy
Code coverage

These should be enabled to run on every push and pull request, regardless of branch.

Support `msd` file format

The msd file format is a Song file format that should be supported.

It has been referred to as the "granddaddy" of many simfile formats, including dwi, sm, ssc, and many other derivatives. Its tag format is common among all of these formats. It is basically a subset of the other formats with a few differences, so it shouldn't be that much more work to support it as well.

A specification of the msd format can be found here (version 2.0β1): https://web.archive.org/web/20070606025338/http://doremi.kalin.to/ddr/msd_format.html

Tags:

Support FNF format

Some prior art: https://github.com/shockdude/fnf-to-sm

Friday Night Funkin may be worth including in this library. Definitely not a high priority, but it could be fun to have.

Add `subtitle` field to `Song`

A subtitle field will store subtitle information for a Song simfile. It should be an owned string.

Define MSRV

A minimal supported Rust version needs to be defined for this library. It can be established when enough features are added that it can be reasonably evaluated.

Add `Difficulty` enum

A Difficulty enum should be added that can encode the equivalent of Novice, Easy, Medium, Hard, Expert, and Edit difficulties. This will be used for both Song and Course structs.

Add `title` field to `Song`

A title field will store title information for a Song simfile. It should be an owned string.

Subtitle Assumption

For dwi and msd formats, subtitles can be assumed from #TITLE tags. SM5 does this using their own form of detection, using a set of delimiters that begin with spaces (see the source). However, it looks like including the spaces is surprising to people (see this issue). It would be better to match without including the spaces.

The matching should be on \t, -, ~, (, and [, in that order, to match the order which SM5 prefers. \t should be removed when splitting, but every other character should be included with the substring.

When splitting, the character split on should be preserved (e.g. if we split on a \t that should be included in the struct) to ensure that the same #TITLE tag can be reconstructed when writing back to a dwi or msd file. This delimiter could be provided as well on Song construction to specify how the joining should be done when constructed from other formats. This way, simfiles with subtitles could be written to dwi format with the subtitle provided as part of the title.

Add `artist` field to `Song`

An artist field will store artist information for a Song simfile. It should be an owned string.

Parsing of `msd`-style tags

Many simfile formats use msd-style tags, which are of the format #TAGNAME:param0:param1:param2; for an arbitrary number of parameters.These should be uniformly parsed for all of these formats to reduce code duplication.

For the most part, this will be straightforward. However, it should be noted that dwi has a #BACKGROUND tag that unfortunately has a few exceptions to this rule, with ; characters being required within the tag's parameters.

Optional Logging

Logging can be optionally provided by an optional dependency on the log crate.

An idiomatic way to provide this logging is to do it similar to how the rand crate does it. They define aliases for each log level that wrap the cfg-gated statements: https://github.com/rust-random/rand/blob/master/src/lib.rs#L63-L92

These should be done in a separate module. Perhaps something like internal_log.

Support `dwi` file format

Song structs should be readable from and writable to the dwi format.

dwi predates sm, and is still supported by Stepmania. Format specification is available here: http://dwi.ddruk.com/readme.php

There is also a largely-incomplete write-up about the format on the Stepmania wiki: https://github.com/stepmania/stepmania/wiki/dwi

Required Tags:

#TITLE
#ARTIST
#GAP
#BPM

Optional tags:

Define `Course` struct

A Course struct needs to be defined to allow writing from and reading to course simfiles.

The scope of this struct's compatibility will be significantly smaller than the Song struct, but it will need to be compatible with both DWI- and SM-style crs files.

Define `Lyrics` struct

Define a struct to contain lyric data.

There exist multiple simfile formats for song lyrics, and this struct would be able to read and write these respective formats.

Text Encoding

It seems that the standard for simfile encoding is UTF-8, but I haven't found any definitive specification of this. Some of these more obscure and old formats may use other encoding formats. I am fairly sure that .sul-based formats used some other kind of encoding, but I have yet to confirm that.

One artifact of interest is a comment in the SM5 .dwi loader code stating that there is no definitive spec for .dwi files, and that some files found are ISO-8859-1. Luckily, ISO-8859-1 is pretty easily converted to UTF-8. If DWI files are encoded differently, it is likely that MSD files are the same, although this again has not been confirmed.

For now things are assumed to be UTF-8. There is no way to deduce from the file alone what encoding format it uses, unfortunately, so the only way we can support additional encoding formats is if a list of encodings per file format is compiled. Ideally, this library will read and write files in their original encoding formats.

Support `ssc` file format

Song structs should be able to be read from and written to the ssc format.

The ssc format is the standard song format for Stepmania 5, and therefore has high priority.

Format documentation is available here: https://github.com/stepmania/stepmania/wiki/ssc

Header tags:

Chart tags:

Cache-only header tags:

Deprecated header cache tags:

#LASTBEATHINT
#MUSICBYTES
#FIRSTBEAT
#LASTBEAT

Add `selectable` field to `Song`

This field will indicate when the song can be selected from the menu.

This should be an optional enum representing all possible values, which are:

While YES and NO are the only options supported in SM5, the other options are still valid and should be preserved. However, SSC files may need to have those values converted to YES when writing, since I don't think SSC officially has ever supported the extra options, and they technically are not valid.

Support `sm` file format

Song structs should be able to be read from and written to the sm simfile format.

sm is the standard file format for Stepmania. While it has less features than ssc, it is still widely used due to its compatibility with so many different programs.

Format documentation available here: https://github.com/stepmania/stepmania/wiki/sm

Header tags:

Time Signature Agnostic Charts

Modern simfile formats support time signature fields. In Stepmania, these are only allowed to have denominators of 4 or any power of 2 above 4. However, we can support any denominator value.

A denominator can be thought of as defining the length of a beat in relation to a whole note. So a denominator of 4 makes a beat the length of 1/4th of a whole note. 8 makes a beat the length of 1/8th of a whole note, and so on. So for non-dyadic values, such as 5, it just defines a beat to be 1/5th the length of a whole note.

Meanwhile, a numerator just defines how many beats there are in a measure. Technically this can be a float (2.5, for example, defines the measure as having 2 and a half beats), but I understand Stepmania only supports whole numbers. Floating point numbers can be implemented to a certain point, as long as the decimal equates to some usable fraction of a beat.

So for representing a chart with custom time signatures, I think things should be thought of in "beats" rather than in "quarter notes, eighth notes, etc". We should have a "beat" (in 4/4 time, a quarter note, but in 4/3 time a triplet note, etc), "half beats", "quarter beats", "third beats", etc. Note that the difference between 4/4 and 4/5 is not the number of beats, but the fact that 4/5 takes 4/5th the amount of time as 4/4 does, which is something that should ideally be handled by whoever consumes the chart information accordingly, although technically the bpm should change to reflect that fact.

For example, a .scc file in 5/8 time will have each eighth note read as a "beat", and there will be 5 of them stored per measure. This will be parsed the same as something in 5/4 time, since there are the same number of beats. So a measure like this:

would be parsed the same in both cases, with the only difference between the two stepcharts being the time signature's denominator.

For older formats, like .msd and .dwi, the time signature will always be 4/4. However, it should be noted that .sm technically supports the #TIMESIGNATURES tag, as although it was removed from the specification years ago, Stepmania's parser still to this day can parse the value and will use it. Oddly, the tag seems to have been reincluded in .scc despite the concerns about the tag that led to its initial removal apparently never actually having been resolved.