devinacker / omgifol Goto Github PK
View Code? Open in Web Editor NEWA Python library for manipulation of Doom WAD files
License: MIT License
A Python library for manipulation of Doom WAD files
License: MIT License
https://github.com/devinacker/omgifol/blob/master/lump.py#L165
The format for the image used ("P") does not support transparency, and any transparent pixels in a graphic are rendered at RGB(255,0,255). This means not only will all exported sprites/patches contain magenta areas, when building textures from patches, every patch with a transparent area will put magenta pixels over the texture.
Example: https://puu.sh/uUJAa/3c86248001.png
A (computationally slow) workaround is possible currently with the following code:
patch_img = wad.patches[patch_name].to_Image()
patch_img = patch_img.convert("RGBA") # Convert to a format that supports alpha transparency
pixdata = patch_img.load()
width, height = patch_img.size
for y in xrange(height):
for x in xrange(width):
if pixdata[x, y] == (255, 0, 255, 255): # Find any magenta pixels
pixdata[x, y] = (255, 255, 255, 0) # Replace with transparent pixel
This adds a significant amount of time to loading patches, especially when working with entire wads of textures such as the IWADs.
This was already done in 301ca31 but got reverted somehow during the python 3 cleanup/merge
When attempting to save a map to the wad, it would error out with
File "mirror.py", line 40, in <module>
if __name__ == "__main__": main(argv[1:])
File "mirror.py", line 38, in main
outwad.to_file(args[1])
File "/usr/lib/python3.6/site-packages/omg/wad.py", line 275, in to_file
self.__dict__[group].save_wadio(w)
File "/usr/lib/python3.6/site-packages/omg/wad.py", line 138, in save_wadio
wadio.insert(t, hs[t].data)
AttributeError: 'list' object has no attribute 'data'
because omg.util.zstrip() is actually kind of broken.
It should possibly be replaced with a function that takes either a str or bytes and returns a string, so that all operations involving lump names are guaranteed to use actual strings and not bytes
Hi devinacker, when I generate the wad file, how can I set the initialized location and the goal location?
Could you offer some clues? Thank you very much.
This code:
import omg
inwad = omg.WAD()
inwad.from_file('doom.wad')
ed = omg.MapEditor(inwad.maps['E4M7'])
gives the following error:
Traceback (most recent call last):
File "e4m7_error.py", line 6, in <module>
ed = omg.MapEditor(inwad.maps['E4M7'])
File "omg/mapedit.py", line 161, in __init__
self.from_lumps(from_lumps)
File "omg/mapedit.py", line 183, in from_lumps
self.sidedefs = self._unpack_lump(Sidedef, m["SIDEDEFS"].data)
File "omg/mapedit.py", line 176, in _unpack_lump
return [class_(bytes=data[i:i+s]) for i in range(0,len(data),s)]
File "omg/mapedit.py", line 176, in <listcomp>
return [class_(bytes=data[i:i+s]) for i in range(0,len(data),s)]
File "<struct>", line 12, in __init__
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc0 in position 4: ordinal not in range(128)
Running omgifol from Arch Linux, Python 3.5.1.
I see why this #27 happens. The acs script is not written into the new wad file. The data is stored in map_editor.scripts.data.
I have tried using the demo/mirror.py to verify this. And it may be the reason why the impassable walls can be passed through.
Could you fix this? Thanks in advance.
https://github.com/GitExl/WhackEd4
Whacked4 is written in python, it might be worth seeing if we can pull features from it to add dehacked support to omgifol
To create a PLAYPAL lump, the process is:
pal.make_bytes()
lump = omg.Lump(pal.bytes)
To create a COLORMAP lump, the process is:
lump = col.to_lump()
The API should be consistent when dealing with different types of data. I think the to_lump() function should be available on the Palette class since that is consistent with most of the rest of the API (to_file, to_Image etc)
WadSmoosh uses omgifol for all its heavy lifting, and I've noticed that the map WADs I extract (from the source IWADs) using it don't end up with the same "map MD5 sums" that GZDoom et al expect for detecting and applying the map fixes specified in its internal data. I'd much rather let GZDoom apply its fixes than add lots of custom fix code to WadSmoosh.
According to its code, each entry in GZDoom's compatibility.txt is an MD5 sum of the map's "header, THINGS, LINEDEFS, SIDEDEFS, SECTORS, and BEHAVIOR lumps". When I try "mapchecksum e3m4" for the omgifol-extracted version versus the doom.wad version, I do indeed get a different result.
The WadSmoosh code for extracting a map from an IWAD looks like this:
def extract_map(in_wad, map_name, out_filename):
out_wad = omg.WAD()
ed = omg.MapEditor(in_wad.maps[map_name])
out_wad.maps[map_name] = ed.to_lumps()
out_wad.to_file(out_filename)
So it seems like something MapEditor.to_lumps() does introduces a change from the input data.
Bug, or feature?
Just for an example, I was using this with v1.4 of CHEX3.WAD, the same one that can be verified with the checksums here: https://doomwiki.org/wiki/CHEX3.WAD
This wad just so happens to be missing a P_END entry, so everything afterwards gets put into the patches group, this includes flats and map lumps that come after all of the patches in this wad.
Give the user the option to export a texture, built from patches, as an Image(). Currently omgifol users have to manually traverse the data and build patches themselves, but this should be an in-built feature of omgifol.
My current implementation (including transparency fix in issue #17) is as follows:
wad = omg.WAD(path)
tex = omg.txdef.Textures(wad.txdefs)
# build the textures out of the patches
for texturedef in tex:
new_img = Image.new("RGBA", (tex[texturedef].width, tex[texturedef].height))
for p in tex[texturedef].patches:
pimg = wad.patches[p.name.upper()].to_Image()
pimg = pimg.convert("RGBA")
# try and fix transparency issues
pixdata = pimg.load()
width, height = pimg.size
for y in xrange(height):
for x in xrange(width):
if pixdata[x, y] == (255, 0, 255, 255):
pixdata[x, y] = (255, 255, 255, 0)
new_img.paste(pimg, (p.x, p.y), pimg)
but this should be as simple as follows:
wad = omg.WAD(path)
tex = omg.txdef.Textures(wad.txdefs)
# get the textures
for texturedef in tex:
new_img = tex[texturedef].to_Image()
This code:
import omg
w = omg.WAD()
w.from_file('doom.wad')
img = w.sprites['HEADA1'].to_Image()
produces this error:
Traceback (most recent call last):
File "image_error.py", line 6, in <module>
img = w.sprites['HEADA1'].to_Image()
File "omg/lump.py", line 169, in to_Image
im.fromstring(self.to_raw())
File "omg/lump.py", line 151, in to_raw
pointers = unpack('%il'%width, data[8 : 8 + width*4])
struct.error: unpack requires a string argument of length 504
Happens with every graphics lump I try in a wad.
Running omgifol from Arch Linux, Python 2.7.11 with PIL version 1.1.7 (Pillow version 3.2.0)
probably allow opening a pk3/pke/etc. the same as a WAD, use LumpGroups for directories, etc.
Currently calling WAD.from_file() on a read-only WAD file fails:
File "/usr/lib/python2.7/site-packages/omg/wad.py", line 256, in from_file
w = WadIO(source)
File "/usr/lib/python2.7/site-packages/omg/wadio.py", line 75, in __init__
self.open(openfrom)
File "/usr/lib/python2.7/site-packages/omg/wadio.py", line 88, in open
self.basefile = open(filename, 'r+b')
IOError: [Errno 13] Permission denied: '/usr/share/doom/freedoom2.wad'
As can be seen at the end of the callstack the file is opened 'r+b', which is read and write.
This is unfortunate in my case since I'm only interested in examining/reading a WAD file. To work around it I copy the WAD to a temporary directory, open that, and then delete it on exit.
I keep my WADs read-only so they are not accidentally modified. Perhaps other people do too, or they should.
Also, WAD.to_file() makes me think there is no need for need for WAD.from_file() to open the file read/write.
Hi,
papers 'RL2: Fast Reinforcement Learning via Slow Reinforcement Learning' and 'A Simple Neural Attentive Meta-Learner' use 1000 randomly generated maps with different initial and target locations. Could you give me light on how to do this?
Thanks very much
Has this been considered yet? It would help with keeping it up to date for users, meaning avoiding issues like Linguica's, using old versions like fredrikj's latest update.
This can be avoided by building nodes and adding the derivative lumps
▶ lswad trivial.wad
name size offset
MAP01 0 12
THINGS 10 12
LINEDEFS 56 22
SIDEDEFS 120 78
VERTEXES 16 198
SECTORS 26 214
WADCSRC 113 240
▶ cat bug1.py
import sys
from omg import *
from omg.mapedit import *
inwad= WAD()
inwad.from_file(sys.argv[1])
ed = MapEditor(inwad.maps['MAP01'])
▶ python bug1.py trivial.wad
Traceback (most recent call last):
File "bug1.py", line 6, in <module>
ed = MapEditor(inwad.maps['MAP01'])
File "/home/jon/git/doom/omgifol/omg/mapedit.py", line 180, in __init__
self.from_lumps(from_lumps)
File "/home/jon/git/doom/omgifol/omg/mapedit.py", line 234, in from_lumps
raise ValueError("map is missing %s lump" % e)
ValueError: map is missing 'SECTORS' lump
also vanilla compatible when possible (https://www.doomworld.com/vb/doom-editing/71482-importing-tall-sprites-to-work-in-vanilla/)
I'm not sure fredrik's original reasoning for creating the struct system for omgifol, but it might help improve development to replace them with custom classes instead.
One thing to look at before attempting this is how fast either of them are. Benchmarking will be needed to see if there are any significant speed differences between the two.
However if it is seen as an unnecessary change then that's reasonable too. The issues I have with the struct system are somewhat minor anyway.
https://github.com/sirjuddington/SLADE/blob/master/src/Utility/Polygon2D.cpp#L95-L127
https://github.com/jminor/omgifol/blob/master/demo/wad2obj.py
Being able to convert a sector to convex polygons is an incredibly useful feature. This can be used for many operations based around converting a map into other formats, or for creating 3d visualisations, increasing the applications of omgifol. Above linked are two possible sources to look at for implementation.
Hi,
in Heretic and Hexen, full-screen grahpics like e.g. "TITLEPIC" are stored in a "raw" format instead of the usual patch format. That is, the lumps consist of 320x200 bytes, each holding a palette index.
Please allow to access them using the wad.graphics[]
API.
Been discussing this with @devinacker, currently you can just read it like any other zip and read into the Lump classes, but there's potentially a way for omgifol to read the structure and just use LumpGroups for directories.
I replaced my old omgifol scripts with these and under Python 2.4.4 they no longer worked properly. "from omg import *" threw up errors.
If it now requires Python 2.7 please update the docs to say so.
CRC32 for STRIFE1.WAD is 4234ACE5
from omg import WAD
from omg.txdef import Textures
Textures( WAD("STRIFE1.WAD").txdefs )
Traceback (most recent call last):
File "pydevconsole.py", line 364, in runcode
coro = func()
^^^^^^
File "<input>", line 1, in <module>
File "omg\txdef.py", line 50, in __init__
self.from_lumps(*args)
File "omg\txdef.py", line 59, in from_lumps
if "TEXTURE1" in g: self.from_lumps(g["TEXTURE1"], g["PNAMES"])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "omg\txdef.py", line 62, in from_lumps
self._from_lumps(args[0], args[1])
File "omg\txdef.py", line 78, in _from_lumps
texture.patches.append(PatchDef(x, y, name=pnames[idn]))
~~~~~~^^^^^
IndexError: list index out of range
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.