mateor / auto-patcher Goto Github PK

Not the clearest description, I know.

The reason I have been struggling so much with the pdroid extension just became clear. Not that I have it licked, but the reason these updated patches have beaten my ass so bad.

When I was calling pdroid_ext on the command line, it was also attempting to apply both pdroid_ext but also regular pdroid as the RID. Which is weird, and should be tracked down. But boy has it caused me pain suffering and loss of sleep.

Even patching the exact rom I made it from was failing. Finally I noticed that even though I had only called pdroid_ext, RID had pdroid_ext AND pdroid listed.

Renaming pdroid_ext to extension has obviated the problem for now. But it needs more investigation into root cause.

Should be relatively easy to introduce.

I believe it fails through not having a build.prop. So we could copy the existing one over, if we wanted, and just include the info that any build.prop changes would be set back to the roms stock state.

Or we could provide the function that could repack the rom with the patched files, which would make patched roms distributable in and of themselves (probably the last thing XDA needs, honestly).

Or, I could finally sort out the array work necessary to associate different BIDs with the PIDs. Which is probably the best thing all around.

Just putting this here so I remember it.

Too big!

I can for sure symlink a bunch of the backup patches from openpdroid to pd2.0. The majority of the extra size seems like it may come from the executables.

I am tired and too close. But two odd things happen...

1)### create updater script ###
inflated: META-INF/com/google/android/update-binary
inflated: META-INF/com/google/android/updater-script
... using Clockworkmod installer ...
inflated: system/etc/ppp/ip-up-vpn <-------------
...adding permissions...

I see where this is ordered in the script, but I am not sure what we are accomplishing. It looks like it interferes with the copy later. Just thought I would point it out.
and

2. at copy stage;
   cp: cannot overwrite directory `system/etc/ppp/peers/gprs' with non-directory

The update.zip doesn't end up having a system/etc/ppp/peers/ directory, even though those files are in common available to be copied.

Cm7 patches aren't working for troubling and indeterminate reasons. The existing patches go through. But I made new patches, and when I apply the patches to the stock rom they have been made from, the rom won't boot, no boot animation even.

If I flash the pdroid-patched build, it boots like it should and pdroid works. But applying the diff from those system.jars breaks booting.

I can't get ADB working to get a log; but unless you have some ideas, that will probably be the end of Gingerbread support.

I had added a SyncStorage provisional awhile back and that worked recently. but no booting now.

The most troubling thing about this is the idea that it is perhaps an indictment of the entire auto-patching process. Why would applying this patch break the ROM?

I even tried modifying the updater-script to its simplest form on the chance that was the cause.

Wsot has a good point. I don't think i need to list ask of the available dates by default anymore. And it will look much better without it. The build query is strong to the point that it is deprecated.

I will leave it as an option, in case people want earlier or specific versions of software. So I need to think of a way to allow it to be called from CL. Like I will be doing with verbose.

Also, I am very close to the final package-style things. If a ROMname is attached to the -h flag, I would like to filter only applicable mods.

Perhaps there is a key to one in the other.

Remember to run the result by KicknGuitar.

On GIT version both command would fail
[ -f "$TOOL" ] || printerror "patch tool is missing"
[ -f "$PTCH" ] || printerror "patch patches are missing"
as autopatcher is looking for files that are actually doesn't exist on GIT version (patch_tools.tgz and patch_patches.tgz)
Another method of verification is needed I guess.

Only needs to be called once per run

There is still some work to be done on provisionals, this may be why we have some failures that are hard to pin down. I am putting together a proper OSX package, and when that update goes out, I will add the provisional copying to the log so we can explicitly see when it happens.

In the paste below, you can see that ContentResolver fails, as always, and I elect to copy it. Provisionals proceeds to services.jar, where ActivityManagerService fails, and I again elect to copy it. But after that the fail menu comes up again for ActivityManagerService, even though I elected to copy it.

But when I agree to copy Actviity again, the operation fails because there isn't a copy of ServerThread in the provisionals folder. It looks to me like The provisional menu pops up in the middle of patching and then goes back to patching, although I don't know why the ServerThread fails as well.

terminal output - http://pastebin.com/ns4rW5Fd
log- http://pastebin.com/VyKx6UzH

You will see in the log that it reports the areas patching failed. I will probably disable patching failure details for Pdroid, as it can get ridiculous. It is merely a test function at the minute, anyway.

We probably just need to reset the {FOO}.log after each provisional copy , I haven't had a chance to look at it carefully yet.

Not necessarily an "issue", but something I think we should have. Since I have three mods aimed at grouper, I intend to start a thread in that device forum. I think we can recruit a bunch of users, and since i own the device, I can easliy maintain the patches.

I would like to offer the Nexus mods separately and as a suite- I don't have any ideas how to accomplish that yet, but I thought I would add it here.

The copy function, if asked to copy over a file that is destined to be compiled into another file (i.e .uventd.grouper.rc.kate-swp in the 3gdongle mod) it will drop the file in its destination folder into the update.zip.

There is a hack work-around at present that fixes the problem, but we eventually need to make the copy file function more versatile.

Workaround: 914d3bf

We didn't include any files in restore for this mod.

I will look at it...

Self-explanatory, I think. Keeping the patches in a web hosted environment makes sense for a bunch of reasons.

• Smaller downloads- we could even add an auto-update line to the script, where the first action would be to check and update the script, making the tool download a one time thing.
• Related to above, but everyone continually downloading the same patches each time is an enormous waste.
• Allow devs to maintain their own patches. I know that trevd with the 3gdongle and OTG is up for it. As we continue to refine our dev tools, this could get easier and easier. Then we could just give write access or ftp privileges to each mod's dev. The ability of devs to jump in and create their own packages easily would be great.

I am going to try and keep up with some of our more administrative duties as the 2.1 branch goes towards stability.

In case of a failed apk patch, the autopatcher should check to see if there is an apk available for copying. For these very small apks, patching is best. But better to copy than to have it fail on account of some small diff inconsistency.

Fully-working as is (ostensibly), but the issues with straight cat'ing the Nook sdcard installs and CM7 updater-scripts has me on alert for future issues with our original method.

The framework for creating our own custom updater-scripts is there with the files.txt and set_perm.sh. We could enlist special_instructions.sh for particularly nasty cases.

You can see my thoughts, and the framework I have in-place to accomplish this at the end of the CM7 issue

As I work on the ABQ (build date query) I have realized it is time to reorganize the patches. I have been laying the seeds with the small changes in where/how I have stored things.

I am going to start migrating the patch system in branch 2.1. This will involve organizing everything by ANDR vesion, i.e. gingerbread, ics, jellybean. I am adding some checks right now to filter the types completely.

With the other improved checks, this SHOULD mean that the only argument people will have to pass, other than MODTYPE, would be a simplified ROMTYPE, meaning {aosp, aokp, cm} (possibly 'pa' as well, if the merge doesn't solve some problems)

It is going to be a bunch of work, though, to redo all those symlinks correctly.

This was requested again. I actually had this for, 1.0? maybe, but it got shelved due to the space in the zip name. I am experienced enough to quickly handle that, so back it goes.

A reminder.

I think I am going to change the script to default use the smali binaries on apks where possible.

The smali binaries are BY FAR better maintained, and they work on all the system apps except framework-res.apk.

As it stands now, we can't interchange the patches easily bacause they default to the different stripping options. But apktool doesn't even have an option to eliminate line numbers, unless I am missing something.

I think adding an apk.txt just like we do for so many other things would be the way to do this. A thought for an upcoming version.

I am moving towards looking at this...let me see how the new smali patch python scripts I was sent work out...if I can patch this stock i9100 rom then we will integrate it and see.

It is going to be a bit of a hassle, as every manufacturer does it differently, and Nexus devices are themselves a whole other story (albeit an easier one). I am putting this here so as to be on the official action list.

Just a general fail safe- not pressing.

But if (say) we just had a patch for PA and no general. If someone tried to patch a CM rom, the patcher would find no match and then begins it's totally bogus unzipping of the rom itself.

Easily solved...but I need to make sure I have addressed it.

I thought it was an error with apktool, but it isn't. After rebuilding apks (on my system) and switching to the build/apk directory, java spits out a "system cannot find path" error and quits updating the jar.

So there is an update.zip created at the end of the script, but it hasn't been updated with the rebuilt apk files. I have checked the apktool and it works fine.

As soon as I get a fix for this, I will commit the new exe binaries and put out a release. Maybe 2.01.1, which is a smalii update to the script and new patches (i.e. the sed patches that pastime just fixed.)

Any ideas what is going wrong? I am just beginning to look at it, but I was mostly working on other things while the cygwin path stuff was being hammered out.

As I work up towards 4.2.1, I am adding the CEILING variable as well, to cap mods I have not brought up yet and just because eventually there will be mods that I either do not want or am not able to carry up to newer versions.

I want to add info blurbs to each of these as well, though. Talk about the devs, functions and perhaps provide some links.

The framework is already done. I would also like the user to be able to get that info from the help menu, so ./auto_patcher -h voice would print out available patches, compatibility information and the blurbs.

This would go hand in hand with the long-promised verbose option. Could be done by modifying what happens when -h is detected as an argument.

apktool part may not be fully functioning yet... needs investigation.

In the Nexus 7 forum, custom kernels are all the rage (because they are so easy to make, IMHO. devs just pump out kernels and builds for the whole nexus line from the same code, and then barely support and of them. Anyway...) and I realize that if a mod requires boot.img patching there is no easy way for them integrate that boot.img into the patcher.

Because if they update their rom with the update.zip they will lose their custom kernel. If they flash the kernel, they would lose 3gdongle, in/secure, or pdroid-aosp.

I propose that we add an option to have their custom kernell end the command line and be intergrated into the update.zip, regardless of whether they choose a mod that entails boot.img patching.

This is a road-map of major to-dos for me to keep an eye on, as well as open up a discussion for collaborators.

• Hosted patch repo.
  This will be done for the next big update, it is time. ALL of the contents of the BID folders will be kept in the cloud, as well as the common folders. I am going to talk with Team Win, I think they'll give me a public folder.
• Universal GUI -Linux, OSX, and Cygwin.
  It won't be very pretty, unfortunately. But I intend to write a GUI in python, perhaps using kivy, perhaps not. The advantage to kivy is that it can be run on Android. I am starting to feel some pressure that we won't be the first to do these things if we don't go ahead and do them.
• Android GUI-
  Could conceivably be derived from the above, we'll see.
• Auto-Patcher branch CLEAR
  I would like to branch the main script and rewrite it, making variable names and comments to make it legible and clear to newcomers. I believe the current state is a barrier to new collaborators- Our variables could use more descriptive names and info about what steps the script is taking and how/why.
• Increase Documentation
  Especially on diff_tools. My dream is for a dev to be able to immediately see what those tools can do and be able to submit and maintain their own patches.
• Split the main script.
  This will probably need to happen as the Auto-Patcher makes its way to app. But the script is long now. I think separating it into incumbent pieces and calling them as children will be faster, easier and more conform to best practices. It will also make debugging way easier. Moving towards objects.

I almost did this for 2.3. But I think we need to choose a license to release this under, and the sooner the better. I am starting to get the feeling of sharks circling. It wouldn't suprise me to see an straight lift of the program be released by someone else in the very very near future.

I decided I wanted to wait for pastime to have a chance to speak to this issue. So let me know if you have any thoughts, I will present some candidates in this space sometime this week.

I would like to add to the error checking and add a conflicts/depends structure.

This is something cribbed from the package managers and i would have to finish my crash course in array handling. But this would prevent the user from attempting to simultaneously apply pdroid and pd2.0, or install tabletUI over a PA rom.

I have the feeling I could do this alongside the Android version floor, but the method has not suggested itself to me yet.

Sometimes when people attempt to patch a rom a mistake, either theirs or ours, causes the auto-patcher to attempt to patch with non-existent patches. I feel like this may be a new problem I intoduced, and if so, well so be it.

In either case, when this happens, the shell essentially believes that the empty patches equals successful patching and then outputs an update.zip with rebuilt yet unchanged files.

My solution is to check for the existence of patches and if they aren't there, printusage.

I am going to work on making our logs better, the kind of just fell into being what they currently are, there are some thing that I sometimes wish I new that I am going to implement.

This issue is here mostly as a reminder to me: the new cat X.log function appears with any printerror, which is not ideal, I may take printerror out of the other error checks or some other solution--

Like I said, I am only typing this up so I remember to address it.

If you run insecure or secure on a ROM that doesn't have a boot.img, it outputs a confusing error.

Segmentation fault ./unpackbootimgEXE...

A message that explains that it fails due to lack of boot.img would be better.

There has been a language barrier, so I haven't been able to convince this person that there is no sensative information in our logs. Using 3gdongle, he can't find init.rc to patch. It is unclear if it is because the bootimg isn't unpacked or a path issue.

But it looks like it might be a path? Either way, he tried 2.0.0 and 2.0.2 and after his third try, I uploaded a patched update.zip. So it isn't his rom, that is for sure...he also tried Windows using the VM Parallels, but I was again unable to ascertain which version of the script he used. I think that maybe the Parallels run was with 2.0.0, with known PATH issues.

Anyway, I will try and reach out to some of our OSX users to determine more.

http://dl.xda-developers.com/attachdl/2a5e0da127ce79a53cbe56b9a7067809/5081f02f/1/4/1/5/6/2/3/log20121019170344.txt

Second time I have seen this recently. This is on gnu-linux, and I don't see java in the path. But the smali binaries are working...

The last time I saw this I figured it might be a bug in (f)aapt, so I just reverted that commit since I was in the middle of the 4.1.2 update, meaning to return to it. But now here it is again.

Anyway here is the log: http://dl.xda-developers.com/attachdl/2a5e0da127ce79a53cbe56b9a7067809/5081f02f/1/4/1/5/6/2/3/log20121019170344.txt

opd needs updated provisionals from the patched packages. very Important.

The actual build date query is pretty lean and mean. I really like that section. But the build.prop query section is a horrible mess. It has just grown out of the various Android flavors and there is a bunch of repeated code.

That needs to be slimmed down. I can still call whatever functions are specific to flavors without that whole mess.

we'll see, this actually is going to take a bit of reorganizing of argv[] reading.

Simply have V = "" and if the autopatcher has an argument -v then have verbose attached to every command that takes it. I could be hacky and attach it to the end, but that isn't standard. Maybe I should go look how other programs handle option parsing.

It will produce a fairly astonishing list of output in some cases, I would think. Considering how easy some people think this program is.

I am working on a separate branch so here is a note about this:

Change line 1076 or so to
FAILS=(grep FAILED ${JAR}.log | sed -e 's/.*smali\///' | sed -e "s/\.rej'//")

Which gets rid of the apostrophe

If the user specifies a build date on the command line, it takes precedence and is used. If there is no matching patch, then it defaults to latest.

We should leave the default to latest that happens later as an error check.

But I would like to also reuse some of the advanced build query to try and make the closest match if that patch date doesn't exist. Not based on a build.prop info, but based off of the available patches. The easiest way is to probably make their manual date something like BDIDA, and then run the build query function regardless.

The build query function could use BDIDA (the command-line argument) to match a patch an argument is given and BDID to match from existing patches otherwise.

Similar to what we do with $EXE.

Nook Color forum is reporting that tabletUI is causing boot loops only when installed through sd card installs. They are using a custom CWM, and that may be the issue,

I have asked for logs, but heaven only knows what I will do with those.

I used the very first v6patcher and the original pdroid patcher to make the Nook Botbrew packages, though, and those worked for sdcard.

Might just be tabletUI. I will look into it.

Need to upgrade the tools as well...

tabletUI, 3gdongle, v6supercharger, pd2.0. [bold remaining]

Not too bad...the only one with probable work for me is TabletUI. Maybe I will finally add the GUI that is floating around for that.

Dang it.

And I have really very little idea of what you were doing in this part. I may have to rewrite this from scratch. An unintended consequence of adding the ANDR version to the patches path.

Still plaguing me...

If users put aosp-jb or mod in the command, it defaults to aosp-mod, I believe.

If using aosp for cm9, it will uses the term aosp-jb, which is wrong.
This is if aosp is designated on the command line. The command line aosp filter needs to only apply for 4.1+.

The aosp naming works well, but needs a going over.

perhaps the various levels of smali.txt should aggregate.

As in if ALL pd2.0 need the GSMservice to be copied I could put that in the top-level pd2.0 folder.

But if just Evervolv needed ContentResolver, I could put just that as a smali.txt inside the evervolv folder.

As it stands right now, the problem is easily worked around by having both the GSM and Content listed in the Evervolv smali.txt. But I think that would be an easy way for them to slip by, and letting them aggregate would be better error checking.

Well, someone called my bluff and made files for Evervolv. Which is great, I am all for user participation. But now i have to figure out how to integrate these new patches.

I just spent the last update simplifying the RomTypes, and do not really want to add another. When i was first sent stuff from users, I intended to make an adjunct archive, available as a separate download, containing the more esoteric patches, which would be detected and integrated by the patcher. But i think the energy for that is better spent on hosted patch repo.

I may just make this a subtype of AOSP. By Fraser the easiest things to do yr would be to make it its own RomType. But i really wnt to maintain the current simplicity of the program.

Just in case anyone is reading these, I thought i would open it up for ideas. This will need to be integrated soon, i i intend to make this patches tonight.

During a test, I discovered a stock rom that did not seem to have a preloaded classes file. Check whether or not A) There was a mistake locating that file or B) An option to copy over preloaded classes if the patch is there and the file is not.

Maybe the preloaded classes are simply different in stock come to think of it. That sounds more likely. The stock file was a deodexed i9100 4.0.4 rom. And adding it as a conditional should still work fine.

Obviously you can see the seeds of this in a recent commit. ed4bf51

I left the dashes in the function so it would not get confused with our own patch sets (BID). But if you change that BDID function to

date -d "1970-01-01 UTC $UTC seconds" +"%Y%m%d"

The output mirrors our patch date structure, i.e. 20120918 or whatever.

I haven't time before I leave town, but there is a way to use this to automatically associate the "best guess" BID instead of defaulting to latest.

The only time this becomes a problem is if the rom maker hasn't clobbered in a long time, but that is increasingly rare.

Well under way. Love you pastime!!!

As of the update to grouper's PA rom yesterday (obviously, never fails) we can not support that particular rom at the moment. It seems they have continued to add their framework changes to framework.jar instead of framework2.jar (which we knew, just never thought about. Otherwise copying framework2.jar over wouldn't work) and now pdroid goes over the method cap again.

I am investigating what we can do. I feel that since we managed to patch aosp-jb, which had the exact same challenges, then we can fix this too.

One idea might just to be to submit a patch to CM moving some more methods over to framework2.jar. They just didn't allow enough space for both us and PA. Which is understandably not their priority, but still, it doesn't hurt them to make their framework more friendly to hackers like us.

I can also begin the work moving some classes into framework2.jar myself. I think supporting PA is important. And I am a little worried about our CM support as well. I wonder how close to the limit they are getting....

=~ operator seems to be a partial match operator. (needs confirm)
[[ system/backup/boot.img =~ boot.img ]] would return true
unlikely, but there could be a potential issue with this type of comparison