Giter VIP home page Giter VIP logo

pycricodecs's Introduction

PyCriCodecs

Python frontend with a C++ backend for managing Criware formats. Although for some tasks, python is used purely.

Supporting

I am running this on Python 3.11, although other earlier versions might work

So far this lib supports in terms of:

Extracting:

  • ACB/AWB (Incorrect filenames in extraction.)
  • USM (Any USM there is)
  • CPK (Most CPK's)

Decoding:

  • ADX (All versions)
  • HCA (All versions)

Building:

  • CPK (All CPK modes)
  • AWB (Anything)
  • USM (VP9 support only, ADX or HCA Audio support, multiple audio streams support as well, VP9/ADX only tested and for sure working!)

Encoding:

  • HCA (HCA Version 2.0)
  • ADX (All versions, any bitdepth, any blocksize, any HighPass Frequence, All encoding versions)

Installation and Usage

To install run

python setup.py install

or alternatively

pip install .

Note: all libs (except ADX) here are standardized to take either a filename/path or bytes/bytearray, so you can swap both.

Also, for audio related codecs, the looping input and output is defined in the metadata, the WAV file will not loop, but it will have a "smpl" chunk in the header, same if you want to encode a looping HCA or an ADX, the WAV must have a smpl chunk.

Otherwise it will loop normally.

Usage:

For ADX decoding and encoding:
from PyCriCodecs import *
# Decoding:
adx_data = open("path_to_adx_file.adx", "rb").read()
wavfilebytes = ADX.decode(adx_data) # Decode will return bytes object containing decoded ADX data as a wav file.

# Encoding:
wav_data = open("path_to_wav_file.wav", "rb").read()
adxbytes = ADX.encode(wav_data) # Returns an ADX file as bytes, check the wiki for more options.
For HCA decoding and encoding:
from PyCriCodecs import *
hcaObj = HCA("filename.hca", key=0xCF222F1FE0748978) # You can change the key, or remove it if the HCA is not encrypted. Key can be a hex string.
wavfile = hcaObj.decode() # Gets you the wav file after decoding.

wavObj = HCA("filename.wav")
hcabytes = wavObj.encode(encrypt=True) # and you will get an HCA file.
# You can provide a key from when initializing, otherwise it will default to the default key, you can also encrypt keyless with keyless=true.
# You can also force disable looping on HCA output by force_not_looping = True.

wavObj.encrypt()
# or
hcaObj.decrypt() 
# Any works, given it can be decrypted or encrypted as an HCA. Would do it. You can also pass a key to ".encrypt()", ".decrypt()" uses the init key. 
For CPK extraction and building:
from PyCriCodecs import *
# Extraction:
CpkObj = CPK("filename.cpk")
CpkObj.extract() # Will extract files to a dir names "filename"
CpkObj.extract_file() # Extract a file from a given filename (or an ID for CPKMode 0)

# Building:
CPKBuilder("dirname", "outfile.cpk", CpkMode=1) # CpkMode is important sometimes, get your target mode by extracting a sample table. 
# Given a directory, it will take that directory as root, and builds a CPK for the directories and files inside.
# Output would be a cpk file as specified.
For USM extraction and Building:

-Note that USM building might be a little bit unstable due to bad code, feel free to open any issues if something did went wrong.

from PyCriCodecs import *
# Extraction:
usmObj = USM("filename.cpk") # or bytes, you can add a key by key="KEYINHEXGOESHERE"
usmObj.extract() # extracts all USM contents in the current directory. You can add a directory with extract(dirname = "Example")

# You can also demux the USM internally and manage with the output bytes all you want.
usmObj.demux() # Then you have access to output property.
usmObj.output # This is a dict containing all chunks in the USM, each key has a value of a list with bytearrays.

usmObj.get_metadata() # Not for the user specifically, but if you want to look at the info inside, this is one way.

# Building:
# Needs at least a video to be able to build one USM file, you can add audio pretty easily too.
usmObj = USMBuilder("filename.ivf", "filename.wav", key=0xKEYGOESINHERE, audio_codec="adx", encryptAudio=True) # Basic USM
# You can add a list of audio paths/filenames as well instead of filenames, and that will be added into audio streams in order.
usmObj.build() # Due to bad code, this is heavy on performance and will take some seconds based of the input files.
usmbytes = usmObj.get_usm() # Will return the USM file as bytes. 
For ACB or AWB extraction:
from PyCriCodecs import *
# ACB Extraction:
acbObj = ACB("filename.acb") # It will attempt to open "filename.awb" as well if there are no sub-banks in the ACB.
acbObj.extract(dirname="dirname", decode=True, key=key) # You can turn off decoding by decode=False.
# AWB Extraction:
awbObj = AWB("filename.awb")
# You can either loop through the audios inside with:
for file in awbObj.getfiles():
    file # file bytes.
    open("file.hca or anything", wb).write(file)

# or you can call the extract function, not advised.
awbObj.extract()

Check the Wiki for my thoughts, plans, more options, and some details as well for documentation.

TODO List

  • Add ACB building.
  • Add correct ACB extraction.

Currently Known Bugs

  • USM Building is not stable, and currently does not work at all because I changed the ADX api.
  • Not important, and might not fix: ADX encoding and decoding at higher bitdepths (11-15) adds popping noise.
  • Some CPK's that has the same filename for every file in the entry will overwrite each other.
  • Probably many more I am unaware of, report if you find any.

Credits

  • vgmstream for HCA code.
  • VGAudio for ADX codec, and HCA encoding, both of which I ported into C++ but modified the ADX lib greatly.
  • K0lb3 for helping a lot with python and Cpython, as well as helping me writing some of the code.
  • bnnm for his various contributions on audio formats, helped me a lot with adding ADX and HCA support.
  • Nyagamon for a lot of what he did for criware formats.
  • donmai and his writeup of CriWare's UTF format.
  • 9th for also helping me with some python knowledge.

pycricodecs's People

Contributors

youjose avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

pycricodecs's Issues

Extracted HCA from USM can't be decoded

When trying to decode a HCA I extracted from an USM (Lost Judgment opening), the audio file can't be decoded:
ValueError: Header decoding error, the header is not a valic HCA header.
I also tried playing the audio file in foobar2000 with vgmstream plugin, it resulted in silence. However, when extracting the USM with VGMToolbox, the resulting HCA was smaller and played properly in foobar2000. PyCriCodecs still couldn't decode it, however.

I noticed the header for both HCAs is the same, the differences happen later on. I attached the two audio files I tested with.
files.zip

Installation issues over PIP

Hello! I've had problems trying to install the WHL package via pip. It seems to be some kind of incompatibility:
ERROR: PyCriCodecs-0.2.0-cp39-cp39-win_amd64.whl is not a supported wheel on this platform.

The install command is:
pip install PyCriCodecs-0.2.0-cp39-cp39-win_amd64.whl

My current version of pip and python is as follows:
pip 22.2.2 from C:\Users\[MY_USER]\AppData\My_Programs\Python\lib\site-packages\pip (python 3.10)

My platform architecture is 64-bit and OS is Windows 11.

I also tried downloading the code from the repository, but I get an error asking to install "Microsoft Visual C++ 14.0 (or later)".
What am I doing wrong?

Extracting audio from a multi track USM combines the tracks

I tried extracting USMs from Judgment, which uses ADX for USM audio. They use 2 audio tracks - one for Japanese audio and one for English audio. The extracted files seem to cut off early, sound laggy and have both tracks combined when played in foobar2000.
If I use PyCriCodecs to decode the ADX, it's just a mess. I attached an ADX file where it happens. If needed, I can also send the USM.

Summary_p02.wav.adx.zip

Doesn't compatible on linux aarch64

Always ended up on failed installation.

OS: Ubuntu 22.04.2 LTS
Arch: aarch64
Kernel: 6.2.1-PRoot-Distro

In file included from CriCodecs/CriCodecs.cpp:4:
CriCodecs/adx.cpp: In function ‘bool Decode(int*, unsigned char*, AdxHeader, int*)’:
CriCodecs/adx.cpp:126:19: error: ‘_byteswap_ushort’ was not declared in this scope
  126 |         int scale=_byteswap_ushort(*(unsigned short *)s);s+=2;
      |                   ^~~~~~~~~~~~~~~~
In file included from CriCodecs/CriCodecs.cpp:4:
CriCodecs/adx.cpp: In function ‘PyObject* AdxEncode(PyObject*, PyObject*)’:
CriCodecs/adx.cpp:276:26: error: ‘_byteswap_ushort’ was not declared in this scope
  276 |                 bscale = _byteswap_ushort(12 - power);
      |                          ^~~~~~~~~~~~~~~~
CriCodecs/adx.cpp:250:17: warning: unused variable ‘md’ [-Wunused-variable]
  250 |             int md = 0;
      |                 ^~
CriCodecs/adx.cpp:251:17: warning: unused variable ‘y’ [-Wunused-variable]
  251 |             int y=0;
      |                 ^
CriCodecs/adx.cpp:225:18: warning: unused variable ‘count_forout’ [-Wunused-variable]
  225 |     unsigned int count_forout = 0;
      |                  ^~~~~~~~~~~~
CriCodecs/adx.cpp: In function ‘PyObject* AdxDecode(PyObject*, PyObject*)’:
CriCodecs/adx.cpp:312:13: error: ‘_byteswap_ushort’ was not declared in this scope
  312 |     data += _byteswap_ushort((header.dataoffset))+4;
      |             ^~~~~~~~~~~~~~~~
CriCodecs/adx.cpp:313:26: error: ‘_byteswap_ulong’ was not declared in this scope
  313 |     header.samplecount = _byteswap_ulong(header.samplecount);

Having trouble with a cpk file from a 3DS game

NOTE: This issue has been fixed.
Only one issue remains for Style Savvy: Styling Star. See my next post for more info.

As this tool seems to be more up to date than others, I thought I might have a better luck here.

I'm having trouble with a cpk file (character_motion.cpk) from a Nintendo 3DS game called Style Savvy: Styling Star.
Whenever I use a newly created one by this tool or others (CriPakTools and YACpkTool), the game is unable to boot, and stays on black screens.

I've tried all of the CPK modes from 1-3 (0 refused to work, due to an error related to the compress flag), with and without compression, and even forcing Tver to CPKMC2.49.32, DLL3.24.00, all with no luck.
In addition, the extraction feature only extracts the first 445 files out of the 3,570 files from the cpk file.

To look into the file, download it here: https://www.dropbox.com/s/gqpagh7fa311fo7/character_motion.cpk?dl=1

Inside the file are only ID-numbered files with no filenames.
The header is structured like this in the file, and doesn't seem to match what's currently in the source code for the CPK modes.

UpdateDateTime
FileSize
ContentOffset
ContentSize
TocOffset
TocSize
TocCrc
HtocOffset
HtocSize
EtocOffset
EtocSize
ItocOffset
ItocSize
ItocCrc
GtocOffset
GtocSize
GtocCrc
HgtocOffset
HgtocSize
EnabledPackedSize
EnabledDataSize
TotalDataSize
Tocs
Files
Groups
Attrs
TotalFiles
Directories
Updates
Version
Revision
Align
Sorted
EnableFileName
EID
CpkMode
Tvers
Comment
Codec
DpkItoc
EnableTocCrc
EnableFileCrc
CrcMode
CrcTable

Asking for DM

Hey Author, could we talk in DM ? Twitter @iseriously_ (or Discord : iSeriously#0001) thanks!

ModuleNotFoundError: No module named 'CriCodecs'

Hi,

I followed the install instructions and tried to run the examples provided, but I got a ModuleNotFoundError.

I tried installing the setup.py in CriCodecs folder, but that also failed for me.

Is there anything I'm missing?

Thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.