richardthombs / scunpacked Goto Github PK
View Code? Open in Web Editor NEWStar Citizen data browser and API
Home Page: https://scunpacked.com
License: GNU General Public License v3.0
Star Citizen data browser and API
Home Page: https://scunpacked.com
License: GNU General Public License v3.0
Hello,
When I try to use the the loader, I got this issue :
Unhandled exception. System.IO.DirectoryNotFoundException: Could not find a part of the path 'E:\SCData\unp4k-v3.3.79-master\3.9.0\Data\Libs\Foundry\Records\scitemmanufacturer'.
I don't have the folder scitemmanufacturer in 3.9.0 or 3.9.1.
It's normal?
The size number is mismatched with the actual size of the component.
Take ARGO MPUV for example:
https://scunpacked.com/ships/argo_mpuv
The main engine ARGO_MPUV_Thruster_Main_Fixed and ARGO_MPUV_Thruster_Main_Joint are size 1 thrusters but they are shown as size 4 on the table
https://scunpacked.com/items/argo_mpuv_thruster_main_fixed
https://scunpacked.com/items/argo_mpuv_thruster_main_joint
Hey,
i'm getting the following Error when trying to parse data from 3.15.1 PTU
EntityService: Cache saved to input folder d:\scdata\3.15.1
Load Localisation
Load Manufacturers
Load Ammunition
Load Insurance
Unhandled exception. System.ArgumentException: An item with the same key has already been added. Key: AEGS_Redeemer
at System.Collections.Generic.Dictionary`2.TryInsert(TKey key, TValue value, InsertionBehavior behavior)
at System.Collections.Generic.Dictionary`2.Add(TKey key, TValue value)
at Loader.InsuranceLoader.Load() in D:\scdata\scunpacked\Loader\InsuranceLoader.cs:line 22
at Loader.Program.Main(String[] args) in D:\scdata\scunpacked\Loader\Program.cs:line 112
PS D:\scdata\3.15.1>
I noticed that for some commodities the values โโof shopBuysThis / shopSellsThis are not correctly indicated. I compared some commodities from shops.json with the ShopLayouts.xml file. For example, an Agricium in Port Olisar should have a sell value whereas in json it has a shopBuysThis value: true
On line
scunpacked/Loader/ShipLoader.cs
Line 346 in 0bf66a0
More informations here:
https://stackoverflow.com/questions/28673269/why-double-parse-throws-formatexception-only-through-rdp/28673285#28673285
In the ships.json file, there is a subsection for each ship called DamageBeforeDetach that looks like this for the AEGS_Avenger_Stalker:
"DamageBeforeDetach": {
"Canopy": 80.0,
"Wing_Right": 680.0,
"Wing_Flap_Right": 10.0,
"WingTip_Right": 20.0,
"Wing_Left": 680.0,
"Wing_Flap_Left": 10.0,
"WingTip_Left": 20.0,
"HullTail_Right": 20.0,
"Right_Tail_Fin_Flap": 10.0,
"HullTail_Left": 20.0,
"Left_Tail_Fin_Flap": 10.0
},
Only one of these properties follow the same key naming convention as the rest of the file. All the rest are in PascalCase, while these seem to be underscore separated PascalCase. This makes parsing harder, requiring manual renames of these particular fields instead of being able to use a generic renaming scheme. Would you mind renaming these fields to match PascalCase like the others in future versions?
Also, many of the key names need to be normalized. In some ships, the DamageBeforeDestruction's "nose" key is upper case, and in others lower case. This trips up strict parsers. Please normalize these.
There are no loading cue on the ship/item page so users can't tell whether the page is empty or not responsive or loading something. Maybe some kind of preloader before the page gets all the data for display?
Hey there - I'm trying to assemble a local copy of the website's database, and while following the readme instructions, I'm getting a path error. Per EntityServices.cs:82, there's supposed to be a path, "Data\Libs\Foundry\Records\entities". No such path exists after I ran unp4k as documented. Instead I've got TagDatabase and associated file, so loader throws an exception and exits.
Any pointers would be appreciated. =)
In FlightCharacteristics, some ships have a String type where most others have a float. This breaks strongly-typed languages. While possible to parse, it makes things much more difficult.
May I suggest producing only a single type per field across all ships/items/etc. It's fine to make them different across different entities, but a single entity should only have a single type. If the "Inifinite" value is needed, might there be another way to represent it besides changing the data type of the field?
It would be great to get a call that lists all the items in a category in an API call.
e.g. https://scunpacked.com/api/items/powerplants
And that will list all the power plants json file URLs and through that I can just loop through them to get what I want.
Can you expose your guidelines for contribution? Also, which dev tools are preferred to run/compile/develop the tool?
Freelancer MAX is under the Heavy Freight category, whereas the Caterpillar is under the Medium Freight category. Both are smaller than some of the Hull series of ships, however relative to eachother, the MAX is lesser in every way.
Hello @richardthombs ,
I've a question about the LocalisationService. Sometimes you use it but not everytime, what's the rule here?
What location is this? Fence_Junkyard_Stanton1_1
SC Version: 3.16.1
Windows Version: Windows 10 Pro 21H2
Loader Version: Latest
I followed the README guide on generating the data from the Data.p4k file, however on attempting to load a specific file, it doesn't like a value. I've attempted to try the instructions however the error is thrown on the same file.
D:\Users\redstonedesigner\SC\Data\3.16.1\Raw\Data\Scripts\Entities\Vehicles\Implementations\Xml\CRUS_Starfighter.xml
Deserialisation failed while parsing Probability value "..6": Input string was not in a correct format.
Unhandled exception. System.InvalidOperationException: There is an error in the XML document.
---> System.FormatException: Input string was not in a correct format.
at System.Number.ThrowOverflowOrFormatException(ParsingStatus status, TypeCode type)
at System.Xml.XmlConvert.ToDouble(String s)
at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderVehicle.Read13_Part(Boolean isNullable, Boolean checkType)
at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderVehicle.Read12_DamageBehavior(Boolean isNullable, Boolean checkType)
at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderVehicle.Read13_Part(Boolean isNullable, Boolean checkType)
at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderVehicle.Read13_Part(Boolean isNullable, Boolean checkType)
at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderVehicle.Read18_Vehicle(Boolean isNullable, Boolean checkType)
at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderVehicle.Read19_Vehicle()
--- End of inner exception stack trace ---
at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)
at Loader.VehicleParser.ParseVehicle(String vehiclePath, String modificationName)
at Loader.ShipLoader.LoadShip(String entityFilename)
at Loader.ShipLoader.LoadFolder(String entityFolder, String shipFilter)
at Loader.ShipLoader.Load(String shipFilter)
at Loader.Program.Main(String[] args
stantonsystem.xml is a treasure trove of positional locations, at least for the planetary bodies. Consider exposing it as a parsed json file.
Additionally, shops are related to their locations through their ID:
I'm a bit new in this domain and I tried to parse files from the .p4k.
I've created the loader.exe with .Net and it seems to work fine.
I have extracted all the required files as indicated in the readme with unp4k.exe.
And when i try to use the loader.exe with the command loader.exe --input=MyInputPath\3.12.0 --output=MyOutputPath\3.12.0-json
it returned an error (I have slightly modified path in the error message for privacy) :
Unhandled exception. System.IO.DirectoryNotFoundException: Could not find a part of the path 'MyInputPath\Data\Libs\Foundry\Records\entities\scitem'.
at System.IO.Enumeration.FileSystemEnumerator1.CreateDirectoryHandle(String path, Boolean ignoreNotFound) at System.IO.Enumeration.FileSystemEnumerator
1.Init()
at System.IO.Enumeration.FileSystemEnumerator1..ctor(String directory, Boolean isNormalized, EnumerationOptions options) at System.IO.Enumeration.FileSystemEnumerable
1..ctor(String directory, FindTransform transform, EnumerationOptions options, Boolean isNormalized)
at System.IO.Enumeration.FileSystemEnumerableFactory.UserFiles(String directory, String expression, EnumerationOptions options)
at System.IO.Directory.InternalEnumeratePaths(String path, String searchPattern, SearchTarget searchTarget, EnumerationOptions options)
at Loader.EntityService.BuildItemDirectory(String folder) in LoaderPath\EntityService.cs:line 150
at Loader.EntityService.Initialise(Boolean rebuildCache) in LoaderPath\EntityService.cs:line 87
at Loader.Program.Main(String[] args) in LoaderPath\Program.cs:line 77
I have checked in the .p4k these folders/files are inexistant and in particular \data\libs\foundry\records\shipinsurancerecord\shipinsurance.xml
.
So with the update of insurance in 3.12 I asking myself if your code is up to date ? Or if I made a mistake.
Thank you in advance for your help.
Just checking if you saw the comments post-close in issue #23
Are there any plans to document the json format you're creating beyond the short description in the bottom of the README.md?
As an aside, I'm digging your project. I was unp4king SC tonight anyways intending to figure out where all the data for ships and items was hidden and having no luck. Your project to the rescue.
Since patch 3.15, ships had personal inventory storage for players items.
Theses values are defined in XML files.
Path : Data\Libs\Foundry\Records\inventorycontainers\ships
Value to fetch <SStandardCargoUnit standardCargoUnits="xxx">
Link the value of __ref="abc123"
to the ship XML Data\Libs\Foundry\Records\entities\spaceships\
Line :
<VehicleComponentParams inventoryContainerParams="abc123" >
Hello,
wondering if V2 format is the upcoming future of the files for this project. Should I look to support those now or stick with the originals?
I noticed that you had the application dockerized in k8s. How about adding a docker file for the compile time/development dependencies as well. That way the system will be secured for ever, allowing anyone to build everything with guaranteed build dependencies.
Basically, make a docker image that, when "run", outputs another docker image for the running application. Such a container is actually also quite useful during development; it is a bit like "make" with all dependencies built in. If you put the development docker-file in git, it also allows you to return to any previously working build in the far future and still be able to trivially build that old version.
It would be useful to include the inverted "shops" tab, so you could find all items in a particular shop rather than find all shops that sell a particular item. Both are useful, and sometimes they might even be useful in conjunction.
Maybe rename the "Items" tab into "Equipment", rename the "Shops" tab into "Items", and add a new "Shops" tab? Or have two text fields under the "Shops" tab, one for item names and one for shop names (letting you search for both in conjunction when needed)?
The background image used on the site can negatively affect load time when it is not cached (first-time visitor or expired cache). There are several ways that might help to minimize the effect:
Suppressors and Compensators seem to be missing from the FPS Gear tab. They can be found in the Shops tab, e.g.
Tacit Suppressor1 (WeaponAttachment) [FPS_Barrel, ballistic_attach, energy_attach]
A friend asked for a list of places to respawn at. I had nothing to tell him haha!
In 3.12, CIG added battery, computer, and life support hardpoints to Gladius, Talon, and Talon Shrike. It will probably be adopted into other ships since Gladius is the gold standard test bed ship and the Talons are new ships.
There seem to be lacking values among the commodities. Tram & Meyers at Cellin sell diamonds at 5.85, have a max of 200000 and a very high refresh rate, yet it doesn't show up in scunpacked at all.
I was looking for the shop location for the Aril Backpack (the blue one) and noticed that the shop search doesn't have it. If you just search for "backpack", a lot of them seem to be missing.
The mining commodities (such as Hephaestanite) seem to be in the shop data, though it is hard to get a summary of them. Perhaps a new Mining tab would be useful?
The website host does not have any text compression enabled. Having text compression such as gzip or broccoli should cut down the file size a lot, reducing the load time for visitor and bandwidth for the host.
Basic weapon damage info is done now.
Still to do:
IFCSParams contains a lot of unsurfaced info:
according to your Website almost all of the ships are displayed with the hardpoint_ping and are all S1
https://www.youtube.com/watch?v=xpvKokenHZA
Based on this youtube video there seems to be greater differences on how these scanners work and which kind of response they have with mining nodes.
In order to make an educated buying decision i'd like to dig deeper and see if there are differences in them, that would explain the results shown in the video.
Did you find any?
Many of the items use a different manufacturer code than what the game uses. It might be more consistent to use the same convention as the game.
Some of the example includes:
Aegis Dynamics: AEG -> AEGS
Klaus & Werner: KLA -> KLWE
Kruger Intergalatic: KRI -> KRIG
@richardthombs Again, thanks for this awesome project. I understand most things will be moving to V2 eventually but since there's no dedicated Manufacturers file in v2 yet, I'm looking at the one in the root. I see that there are many duplicates of various manufacturers with different references. What is the significance 'reference' field? I expect that will explain why are there so many duplicates.
/u/malogos already has http://starcitizendb.com/ which is also known as scdb
in some circles, and it also has an api, and has been well maintained for a few years now.
If you're not attached to the name, it may be an idea to rebrand before you go wide/public - perhaps scapi? Could even spin things a bit into "SCnAPI", (Star Citizen Normalized API) and have a little crocodile as a mascot/logo etc =D
Hello everybody,
first i want to thank you for this Project. I'm currently working an an small German Community page Link myself and i'm always coming back to your site to check different informations. I really like the "easy-to-understand"-Structure especially when i dont know the exact title of an Item. All-in-All ....a great site & easy to navigate.
Would it be possible to add a Picture to the Detail Pages of the different Items? I know this could potentially use-up a lot of Webspace & traffic.
At the moment i'm searching a couple of Pictures for different Items. My first Thought was:
"No problem, SCunpacked has everything i need...."
This was the moment where i realized that in the Months before i mostly came here to get "text-Information"about "Magazine Size oder rate Of Fire....for example.
I think it would be Great if SCunpacked were "The Place to go" to get an Image for every Item.
This is the main reason why i'm asking. :)
btw. I added your Project to our Link-Database: Link
Greetings from Germany
(Sorry, if my english is a little rough)
Are the ship sale and rental prices and locations in the data somewhere?
Hi,
With the 3.15.1 build, I have this issue with the 2 Ares files (inferno or ion) :
D:\unp4k-suite\Data\Scripts\Entities\Vehicles\Implementations\Xml\CRUS_Starfighter.xml
Unhandled exception. System.InvalidOperationException: There is an error in the XML document.
---> System.FormatException: Input string was not in a correct format.
at System.Number.ThrowOverflowOrFormatException(ParsingStatus status, TypeCode type)
at System.Double.Parse(String s, NumberStyles style, IFormatProvider provider)
at System.Xml.XmlConvert.ToDouble(String s)
at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderVehicle.Read13_Part(Boolean isNullable, Boolean checkType)
at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderVehicle.Read12_DamageBehavior(Boolean isNullable, Boolean checkType)
at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderVehicle.Read13_Part(Boolean isNullable, Boolean checkType)
at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderVehicle.Read13_Part(Boolean isNullable, Boolean checkType)
at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderVehicle.Read18_Vehicle(Boolean isNullable, Boolean checkType)
at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderVehicle.Read19_Vehicle()
--- End of inner exception stack trace ---
I didn't find what value is missing in the files.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.