project-artist / artistgui Goto Github PK

Before we delete the original oat when recompiling an app, we probe its user and group to later be able to set those accordingly for the freshly compiled, new oat file. However, if compilation fails once (e.g., during debugging) or there is no oat (e.g., we removed it with #10 or the system never created it in the first place), we cannot set user and group properly and the compiled file might be ignored by art.

To fix this, we should be able to determine the correct user and group without having to query the oat each time. This can be solved partially by probing the oat file once in the beginning and persisting the result. However, a superior solution would be to learn per Android OS version (!) what the user and group should be, for example by asking a middleware service such as the package manager or AMS.

Given that we have ready-made ARTist versions at https://github.com/Project-ARTist/ARTist/releases, it would be nice to directly download the correct ARTist version into ArtistGui instead of requiring it to be compiled into the app as an asset. Effectively, this allows to distribute a slim apk on platforms where there is an ARTist release available.

We should still keep around the possibility for developers to work with self-baked versions so that those have a higher priority than the downloaded ones (if needed). Maybe control this via a settings switch?

The app remembers information about instrumented apps. After uninstalling a package the app is still marked as instrumented and present in the db.

In the new version of the UI, we dropped the setting to activate merged apk backup.
Instead, it would be nice to always backup the last merged apk and overwrite the previous one.
This way, we make sure to not fill up too much space but still be able to debug efficiently.

With #8 in mind, we want to be able to 'undo' the instrumentation of applications.
The most straightforward way might be to simply delete the new oat file. When executed afterwards, the app will either be compiled again by the system dex2oat or simply interpreted. In any case, the instrumentation should be gone.

Continuous integration might help to catch bugs, such as:

• version differences (unavailable APIs due to SDK)
• broken assumptions
• problems with dependencies
• UI misbehavior

In the best case, testing should include building the project and running the tests on all supported Android versions. This should be a requirement for landing a pull request.

Travis CI offers free plans for open source repositories, might be worth a look.

When updating the constraint-layout dependency from 1.0.2 to 1.1.2 and importing a module into ArtistGui, the module is not always shown when tapping on an app. While the list of installed modules appears for some apps, it does not for, e.g., the heise online app (which we use for testing quite often).

Since appIcon already can be null, just catch this Exception too. Might be nicer to return the default-Icon.

In the Settings Activity, the back button in the top left corner is broken.
When navigating to this Activity from the drawer, the arrow will not, as intended, close the Settings Activity but add another instance of the Settings Activity on top.

In the context of Project-ARTist/meta#3, we will have different module packages instead of one compiler version that has them hard-coded. So we need to manage those module packages and keep track of them.

Assuming we have a fixed package format, we need logic to

• load modules from the file system and parse them (extract meta information & files)
• persist them in a database
• remove them from the database
• update them from the file system

This issue only focuses on the handling logic, not the corresponding UI.

The primary action of the modules screen is to import a module where a FAB is suitable. A module has metadata like the description which isn't showed anywhere. A module developer can inform the user about something there. That's why we need a detail view for each module.

As soon as we have the module management implemented (#54), users should be able to pick the modules that they want to be applied for their apps. The goal is to extend the compilation UI with a module chooser that allows to decide for modules and their ordering (in some scenarios, the ordering matters). The keep-instrumented option also needs to be extended so that the same set of modules is used for the recompilation after updates. For cases where this is not possible (i.e., module not available anymore), the user should be informed to pick a new combination. A simple hint that the auto update did not succeed and she needs to pick a new combination should be fine.

Promote Dexterous to a standalone tool so that it can also be used outside ArtistGui.

In Travis we run ./gradlew build connectedCheck (line 43). "build" builds the app and runs the unit tests. "connectedCheck" executes the instrumentation tests on an Android device (physical) or an emulator. However we don't create/start an emulator.

Another problem is that Travis CIs highest supported API level is 25 (ArtistGui supports higher versions).
The main problem is that Travis updates/installs its components using the deprecated android binary (in the sdk). I updated the android-sdk cookbook and opened pull request (travis-ci/travis-cookbooks#992).

No ARTist version, meaning dex2oat, libart-compiler.so and the like, is present when ArtistGui is freshly checked out. Hence, we need to check for and warn about missing ARTist binaries.
In the best case, we explain what the problem is and point the user to, e.g., the website.

PackageUpdatedReceiver does not filter the incoming package names and hence starts instrumentation tasks for every single updated package no matter whether we ever instrumented it before.

We should, however, consult the database and check if we instrumented the app before AND the keep instrumented option is enabled.

Whenever an app is updated, we need to detect this and automatically re-instrument after the installation has finished because the installation triggers a new compilation that replaces our custom oat file for this app.

While logging verbosity can be set for ArtistGui itself, it does not carry over to the compiler.
Also, as of now, we seem to create log files for each compilation and never delete them ever again.

We want to have a log management that incorporates and controls the logs and their verbosity from ArtistGui AND the artist compiler, takes care of backups and deletes old logs according to a policy. It might be beneficial to allow the user to control this to a certain extent, e.g., decide on the log rotation schedule.

Exposing the new module management capabilities (#54) to the user requires new UI components and updates to existing ones. In particular, the following is needed:

• module overview UI that displays installed/imported modules
• module details UI that displays available information about a module, such as its version, maintainer, compatibility, ...

The service moves itself to the foreground and creates a notification. In order to reinstrument apps after updates ( #2 ) we start the service from a broadcast receiver which listens for android.intent.action.PACKAGE_REPLACED ( #22 ). After compilation the notification stays in the notification bar.

Port and test on Android 8.0 Oreo

We react to updates (#22) with a broadcast receiver which listens for android.intent.action.PACKAGE_REPLACED. This doesn't work in oreo anymore (https://developer.android.com/about/versions/oreo/background.html#broadcasts). We can't listen for implicit events.

Hi,
I recently browsed through the FOSS app store called F-Droid [0] and I noticed that the ArtistGui app isn't available for download on there. F-Droid is an alternative app store for people that do not use Google and/or the Playstore, so it would be really cool to see your app there.

As F-Droid focuses on FOSS app, they have strict rules of what an app is allowed to do and what not [1], but I believe that the ArtistGui should be fine (but I haven't looked at your source code tbh).

To submit the app to the central F-Droid repo, the fastest way is to clone the metadata repository and create a PR with the new metadata-text file. [2]

Best regards,
gehaxelt

[0] https://f-droid.org/en/
[1] https://f-droid.org/en/docs/Inclusion_Policy/
[2] https://gitlab.com/fdroid/fdroiddata/blob/master/CONTRIBUTING.md

The idea is to redesign the compilation UI to have a fragment/dialog for each app that shows, e.g., whether or when it has been instrumented the last time, and provides functionality to remove instrumentation (#10), reinstrument now or always reinstrument after updates (#9).

After importing a module into ArtistGui it can be selected to instrument apps. However there's no way to know with which module a given app was instrumented. We only know if an app is instrumented. This is needed for #70 (Reinstrumentation after app updates broken)

The PackageUpdatedReceiver is declared in the AndroidManifest.xml and listens always for android.intent.action.PACKAGE_REPLACED. Using PackageManager.setComponentEnabledSetting()¹ we can enable the BroadcastReceiver only when at least one app is marked as "keep instrument" instead waking up ArtistGui at each app update.

¹ https://developer.android.com/reference/android/content/pm/PackageManager.html#setComponentEnabledSetting(android.content.ComponentName, int, int)

There is no way of seeing if an app is instrumented. It'd be nice to see directly in the app list whether an app is instrumented.

The app list can be really long. Fast scroll (like in the contacts app/home launcher) makes is easy to jump to a letter (alphabetically) easier.

The InstrumentationController has a queue to handle multiple instrumentation requests but there is no way a user can see how many instrumentation requests are existing. For a while the progress dialog is empty after touching instrument if there is already an instrumentation (triggered by an app update) going on.

In order to fix the empty progress dialog #32 when the service is handling multiple instrumentation requests we need to cache the latest progress update. Otherwise we can't display the status after the broadcast was fired.

When I implemented the AppDetailsDialog I had to remove the snackbar. It wasn't possible to move the snackbar into the dialog because it was displayed above the dialog buttons and textviews. It's possible to show a Toast.

chmod'ing everything to 777 is an obsolete and dangerous relic from the early times of the project where it was all about getting things done quickly. Now that we want to use ARTist in production systems, we should take care of assigning proper access rights to executable files, such as the resulting instrumented oat file.

The objective is to either dynamically find the correct access rights or find fixed ones that work on a multitude of devices. While the dynamic way is preferable, #11 has shown that it might not always be possible to implement this (easily).

I would assume this is a good first issue if you want to get started with ARTist since you do not have to understand much about the ARTist ecosystem. It is just about finding a way to assign proper access rights to those files that are currently chmod'ed to 777.

DexMerger's current implementation uses a hacky heuristic to detect which dex file is the codelib:
During development, our codelibs were always pretty small, hence the check here tests for the method ID count.
Obviously, this does not work for bigger codelibs and hence we need to replace it.

We can hardly detect codelibs in a generic way since, as of now, they do not have a common format.
Therefore, it might be easier to provide the codelib explicitly and store this information throughout the process.

It's hard for the user to find a specific app in the list. A search for the app name or package name is more user friendly.

ARTist/ARTistGui Support for Android x86/x86_64 Emulator (ranchu)

Needs changes in:

• buildscripts
• architecture & platform probing in ARTistGui's functions
• dex2oat's commandline flags

ArtistGui should import a module, instrument an app and start it triggered by an explicit custom broadcast. This is needed in order to auto deploy modules from an IDE.

Relates to Project-ARTist/meta#21

The app launches the home fragment. It contains a little description and shows information about the git status (current commit, Artist versions, ...). It'd be more useful to show additionally recently instrumented apps, favorite apps (starred in app list) and other things like these.

The reinstrumentation after app updates doesn't work since the introduction of modules. Currently we don't map modules to apps, so we don't know which app was instrumented using which module.

There are situation where multiple modules want to modify the end of the same method. If one module injects a return statement first the changes of the second module will be ignored.

The ModuleImporter is very naively implemented. It hopes the import succeeds always. The importModule(...) catches IOExceptions and returns null which lead to a NullPointerException in the AsyncTask. The exceptions get logged but the user only sees that the app crashed.

Problems:

• Check if module imported correctly in AsyncTask iterating over the result (null -> NullPointerException)
• During import the module lies in a tmp_extraction folder which get renamed afterwards (don't ignore result, it can fail)
• Handle low space or no space
• ManifestParser can't handle missing/unknown json attributes (app crashes)
• The zip can be corrupted or not a zip at all
• Expected files can be missing in zip
• Notify user that something went wrong (be specific but user friendy)

With #8 in mind, we want to have a per-app setting to activate automatic recompilation of apps immediately after they have been updates (and hence the previous instrumentation got lost).

This implies a persistent setting and means to detect updates and trigger compilation accordingly.

It'd be nice to have the an option to import a module before instrumenting an app (during module selection). It's basically a shortcut.

When we install an application (e.g., via adb) while the view displaying installed apps is already in the foreground, the new app does not show up and we need to close the view and reopen it to see the change.

It would be nice to have this view auto-refresh as soon as an application is installed.

If the package name of the instrumenting app is too long the progress text could be cut off (below cancel button).

We mainly work on Nexus and Pixel devices where oat files are stored in the same location. Other vendors might have changed the folder structure, hence we should test whether ArtistGui works there or if we need to add other oat folder alternatives or even a routine to actively search for the oat path.

Currently the BroadcastReceiver (AppListChangedReceiver) which listens for Intent.ACTION_PACKAGE_ADDED and Intent.ACTION_PACKAGE_REMOVED calls Loader.onContentChanged(). This calls forceLoad() on the AsyncTaskLoader. It'd be better to update the existing list instead of rebuilding the entire list.

The ModuleImporter blindly overwrites other modules if both package names match.

We never touch the original apk so the option to replace the the original apk is useless. The setting doesn't exist anymore but LibArtist checks this.