Giter VIP home page Giter VIP logo

saintcoinach's Introduction

Saint Coinach

A .NET library written in C# for extracting game assets and reading game assets from Final Fantasy XIV, now with support for including the Libra Eorzea database.

Functionality

Fully implemented

  • Extraction of files from the game's SqPack files based on their friendly name (or by Int32 identifiers, if preferred).
  • Conversion of the game's textures to System.Drawing.Image objects.
  • Parsing and reading from the game's data files (*.exh and *.exd).
  • Decoding of OGG files stored in the game's pack files (some of the *.scd).
  • OO-representation of the most important game data.
  • Self-updating of the mapping between game data and their OO-representation, in case things move around inside the game's files betwen patches.

Partially implemented

  • Decoding of the string format used by the game. Will return a good string for most queries, but more advanced things like conditional texts are not supported.
  • Parsing of model data works for most models but is still incomplete.
  • Inclusion and parsing of data from the Libra Eorzea application.

To-do

  • Support for audio formats other than OGG.

Usage

Set-up

Note: When building an application using this library make sure to include a copy of SaintCoinach/SaintCoinach.History.zip in the application's directory. This should be done automatically if the project is included in the solution and referenced from there.

All important data is exposed by the class SaintCoinach.ARealmReversed, so setting up access to it is fairly straightforward.

The following is an example using the game's default installation path and English as default language:

const string GameDirectory = @"C:\Program Files (x86)\SquareEnix\FINAL FANTASY XIV - A Realm Reborn";
var realm = new SaintCoinach.ARealmReversed(GameDirectory, SaintCoinach.Ex.Language.English);

It's that simple. It is recommended, however, to check if the game has been updated since the last time, which is accomplished like this, in this example including the detection of data changes:

if (!realm.IsCurrentVersion) {
    const bool IncludeDataChanges = true;
    var updateReport = realm.Update(IncludeDataChanges);
}

ARealmReversed.Update() can also take one additional parameter of type IProgress<UpdateProgress> to which progress is reported. The returned UpdateReport contains a list of changes that were detected during the update.

Accessing data

Game files can be access directly through ARealmReversed.Packs, game data can be accessed through ARealmReversed.GameData.

Game Data (XivCollection)

Specific collections can be retrieved using the GetSheet<T>() method. Note: This only works for objects whose game data files have the same name as the class. This applies for most classes directly inside the SaintCoinach.Xiv namespace, so there should be no need to worry about it in most cases.

Special cases are exposed as properties:

  • BNpcs: This collection contains objects of type BNpc that include data of both BNpcBase and BNpcName. Note: Only available when Libra Eorzea data is available.
  • ENpcs: This collection contains objects of type ENpc that include data of both ENpcBase and ENpcResident.
  • EquipSlots: There is no actual data for specific equipment slots in the game data, but having access to them makes things more convenient, so they're available here.
  • Items: This collection combines both EventItem and Item.
  • Shops: This collection contains all types of shops.

The following is a simple example that outputs the name and colour of all Stain objects to the console:

var stains = realm.GameData.GetSheet<SaintCoinach.Xiv.Stain>();

foreach(var stain in stains) {
    Console.WriteLine("#{0}: {1} is {2}", stain.Key, stain.Name, stain.Color);
}

Notes

State of documentation

Only SaintCoinach contains documentation, and even that only in some places (anything directly in the SaintCoinach namespace as well as some things in SaintCoinach.Xiv.*), everything else is virtually void of documentation.

There should, however, be enough documentation available to know how to use the library for game data, but figuring out the internal workings might prove difficult.

SaintCoinach.Cmd

The project SaintCoinach.Cmd is a very basic console application that can be used to extract various assets. The following commands are currently supported:

  • lang: Displays or changes the language used for data files. Valid arguments are Japanese, English, German, French. If no argument is supplied the currently used language is shown.
  • raw: Exports a file from the game assets without any conversions. The argument should be the friendly name of the file.
  • image: Exports a file from the game assets as a PNG-image. The argument should be the friendly name of the image file.
  • ui: Exports one or multiple UI icons as PNG-images. The argument can either be the number of a single UI icon, or the first and last number for a range of icons seperated by a space. Valid numbers are in the interval [0, 999999].
  • exd: Exports all or a specified number of game data sheets as CSV-files. Arguments can either be empty to export all files, or a list of sheet names seperated by whitespace.
  • rawexd: Exports all or a specified number of game data sheets as CSV-files without post-processing applied. Arguments can either be empty to export all files, or a list of sheet names seperated by whitespace.
  • bgm: Exports all sound files referenced in the BGM sheet as OGG-files.

Godbert

Godbert is a simple application to display game data and 3D models from Final Fantasy XIV using the above-mentioned library.

Functionality

Fully implemented

  • Display of game data / text.
  • Rendering of equipment, including ability to dye (if the item supports it.)
  • Rendering of monsters.
  • Rendering of demi-humans (non-playable races with exchangable equipment.)

Partially implemented

  • Rendering of the game's areas. Only static and completely passive objects are displayed at the moment, anything that's more complex is not included.

To-Do

  • Add more things to show.
  • Store active language in config.

saintcoinach's People

Contributors

rogueadyn avatar ufx avatar vekien avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

saintcoinach's Issues

System.ArgumentException in DataSheet.cs

It is possible to throw a System.ArgumentException in DataSheet.cs on the line:

[code]_PartialSheets.Add(range, partial);[/code]

Full exception:
"System.ArgumentException: An item with the same key has already been added.
at System.ThrowHelper.ThrowArgumentException(ExceptionResource resource)
at System.Collections.Generic.Dictionary2.Insert(TKey key, TValue value, Boolean add) at SaintCoinach.Ex.DataSheet1.CreatePartialSheet(Range range) \SaintCoinach\Ex\DataSheet.cs:line 117
at SaintCoinach.Ex.DataSheet1.CreateAllPartialSheets() in \\SaintCoinach\\Ex\\DataSheet.cs:line 108 at SaintCoinach.Ex.DataSheet1.ContainsRow(Int32 row) in \SaintCoinach\Ex\DataSheet.cs:line 136
at SaintCoinach.Ex.MultiSheet2.ContainsRow(Int32 row) in \\SaintCoinach\\Ex\\MultiSheet.cs:line 102 at SaintCoinach.Xiv.XivSheet1.ContainsRow(Int32 row) in \SaintCoinach\Xiv\XivSheet.cs:line 118

Equipment deformer

  • Look into deforming of equipment when no race/tribe-specific model exists. Possibly in chara/xls/charadb/EquipmentDeformerParameter/c####.eqdp or chara/xls/boneDeformer/human.pbd.

Suggestion: Dump Orchestrion Music

It would be really awesome if SainCoinach.Cmd had an option to dump the Orchestrion music, which looks like it's all at music/ffxiv/Orchestrion/* . For your sanity I'm referring to the Godbert dataset using bold, while the column names within each dataset using italics. For this, having a command like "omusic" would probably be the best.

Here's how this should work:

  1. Create a directory in the output folder for each Name in OrchestrionCategory
  2. Iterate through OrchestrionPath using a for loop, which will give you a Key value to use in Orchestrion, OrchestrionPath, and OrchestrionUiparam. Here's how that value should be used to generate data, path, and file name:
  • Data is File in OrchestrionPath. If there's no File value, log and skip it.
  • Path is OrcehstrionCategory in OrchestrionUiparam, which should be one of the directories created in step 1
  • (optional) A prefix for the file name can be generated Order in OrchestrionUiparam which should recreate a similar file structure to the Orchestrion UI panel ingame. This should be padded out to three digits (ex: 1 becomes 001) which will cause almost all OS's to sort everything correctly, while values that are 65535 (which affects Seasonal and Store Scrolls) should not be used.
  • File name looks like it's Name and in Orchestrion

So assuming I'm understanding all of this correctly, the resulting output should be a rather user friendly set of files that would be organized similar to what's seen ingame.

Animations

  • Look into animations.

Example files:

  • chara/monster/m0099/animation/a0001/bt_common/resident/monster.pap
  • chara/human/c0501/animation/a0001/bt_common/emote/fume.pap

Character customization options

  • Add a structure to Viewer for customizable colour choices (hair, skin, etc.) that can be used across models.
  • Add UI to adjust those colours in Godbert.

Extracting Available SFX?

Hi! I wonder if it's possible to extract the sfx in the game? Currently I could only get the bgms but I'm also interesting in a couple of specific sounds in the game. If it's not supported yet, could it be in the future? :)

Failed to export some .fbx

Hi guys.

I recently tested latest version of Godbert but I see the program failed to export some newest enemy, for examples:
m0677 , m0681, m0650.

"The export of X.fbx has failed"

Shader characterglass.shpk

  • Add support for the shader characterglass.shpk (used by models with glasses, such as Scholar's Mortarboard.)
  • Make the glass opaque (for now) because transparency just makes things more difficult than they have to be.
  • Full support for transparency is going to require major changes to the rendering system so it always renders from back to front.

Lgb files

  • Look into *.lgb file entries other than the only currently known static model type.

Invalid mesh data

Some models in bg/ have a 0 for MeshHeader.VertexDataPartCount, indicating they have no vertex data at all, leading to an exception when attempting to read vertices.

Example file keys (using IndexSource)

  • 0xF6C42306
  • 0xF1A253D6

Incomplete read in ModelDefinition

Example file bg/ffxiv/sea_s1/pvp/s1p1/bgplate/0001.mdl does not reach end of the definition when reading, most likely contains data not present in most models.

Other examples are files in bg/ with keys (using IndexSource) 0x014F7297 or 0x0004EF5F.

[Question] Magic bitwise operations

Hi! Any chance you could elaborate on this code:

private static int CalculateTarget(EorzeaDateTime time) {

I stumbled upon this fascinating piece yesterday, still trying to understand what exactly is being done there:

var calcBase = (totalDays * 0x64) + increment;

var step1 = (calcBase << 0xB) ^ calcBase;
var step2 = (step1 >> 8) ^ step1;

return (int)(step2 % 0x64);

Thanks

Create object of "File" from file on hard drive, not from "SqPack".

Is it possible to create a "File" not from a SqPack, but from external file on hard drive?

I want to edit skeleton of character (change length between some bones - for example "neck" and "head") and then export animation as FBX. In order to achieve it, I think the only thing I need is object of "File" class, which is created from edited by me file (which is stored somewhere on a hard drive).
For example:
D:\skl_c0101b0001_edited.sklb

Does anybody know how it might be achieved?

Shader bgcolorchange.shpk

  • Add support for the shader bgcolorchange.shpk (used by dyeable furniture, such as Riviera Round Table.)

Failure to read ModelDefinition

The ModelDefinition of some models in bg/ attempt to read out-of-bounds.

Example file keys (using IndexSource)

  • 0x9985F796
  • 0x9D0DF859
  • 0x8F8B9215

Performance boost

I wish I had known about this project earlier as I've got my own library but its quite a bit dirtier, but its also quite a bit faster. You can get some really huge performance gains using a few tricks.

https://gist.github.com/anonymous/8689c40a8ef75f5ad30e

I've begun working on incorporating this stuff, but it looks like the structural changes would be significant. Would you be interested including this stuff in the library?

Skeletons

  • Look into skeletons (such as chara/human/c0101/skeleton/base/b0001/eid_c0101b0001.eid, phy_c0101b0001.phyb, skl_c0101b0001.sklb, and skl_c0101b0001.skp) and if they're even of importance.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.