microg / nominatimgeocoderbackend Goto Github PK

With all the effort this project puts in data privacy, choice and independence I don't understand how the requests are still made in plain text for anyone to sniff, log and modify.

Anyone can verify it by taking a cursory look.

I wasn't expecting certificate pinning, but this is a bit silly. Even Google uses end-to-end TLS.

Affected app
Name: [e.g. WhatsApp]
Package id: [e.g. com.whatsapp]

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

System
Android Version: [e.g. 10]
Custom ROM: [e.g. LineageOS 17.1]

microG
microG Core version: [e.g. 0.2.13.203915]
microG Self-Check results: [e.g. All ticked]

Additional context
Add any other context about the problem here.

Thank you!

Thank you for your project and software!

Thank you for maintenance in the past and in the future!

Kind regards and season's greetings!

Describe the bug
As I see it reported throughout multiple issues here and comments elsewhere, it seems this backend is no longer working as expected. Debugging a request to it yields no results. Are there any plans to make it working again?

To Reproduce
Steps to reproduce the behavior:

Please see the above link. As I cannot find the backend's check in the microG self-test and, as a user, have no clue how to give proper steps, that's all I can say: there seem to be no results. I've watched the network traffic and did not even see any network requests being made when the Android Geocoder class was being called.

Expected behavior
A network request is being made, and the reverse-geocoding results are returned.

Screenshots
n/a

System
Android Version: 10
Device: Shift6mq
Custom ROM: ShiftOS-L 3.9 with microG installed "in UserSpace". All checkmarks in the self-test are green.

Additional context
See the linked issue. The gallery app Aves has the feature to group photos by location. For that it needs to reverse-geocode the coordinates as described above.

I installed this app from F-droid. Someone gave me an advise to install this to fix a problem with the GPS on my Fairphone 3 with e/OS.
I was supposed to help getting a better GPS signal but it does not help. How do I uninstall this? I can't find it in the app list.

Thank you very much for your help and time

Affected app
Name: [e.g. WhatsApp]
Package id: [e.g. com.whatsapp]

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

System
Android Version: [e.g. 10]
Custom ROM: [e.g. LineageOS 17.1]

microG
microG Core version: [e.g. 0.2.13.203915]
microG Self-Check results: [e.g. All ticked]

Additional context
Add any other context about the problem here.

Describe the bug
When Nominatim Geocoder is enabled in microg, it sometimes crashes. Moreover, NLP frequently stops working.

System
Android Version: 12
Custom ROM: LineageOS 19.1 (LOS for MicroG)

Additional context
Seen from logcat:

java.lang.NullPointerException: Attempt to get length of null array
	at java.lang.StringFactory.newStringFromBytes(StringFactory.java:46)
	at org.microg.nlp.backend.nominatim.BackendService$AsyncGetRequest.retrieveString(BackendService.java:288)
	at org.microg.nlp.backend.nominatim.BackendService.getFromLocation(BackendService.java:122)
	at org.microg.nlp.api.GeocoderBackendService$Backend.getFromLocation(GeocoderBackendService.java:73)
	at org.microg.nlp.api.GeocoderBackend$Stub.onTransact(GeocoderBackend.java:63)
	at android.os.Binder.execTransactInternal(Binder.java:1179)
	at android.os.Binder.execTransact(Binder.java:1143)

I receive an error when I attempt to enable it in microG settings. Never allowing it to be set up at all.
Any idea what might be going on?

Hello :)

Is it possible to use a sort of offline database ?

Thanks

It crash on KitKat.
I'm using the version 1.2.1 since I haven't found a compiled version of 1.2.2.

Possibly related to #19 but I'm not sure.

[ 05-23 02:35:48.973  1892: 2116 I/ActivityManager ]
Start proc org.microg.nlp.backend.nominatim for service org.microg.nlp.backend.nominatim/.BackendService: pid=3735 uid=10049 gids={50049, 3003}

[ 05-23 02:35:49.090  3735: 3735 D/ActivityThread ]
handleBindApplication:org.microg.nlp.backend.nominatim

[ 05-23 02:35:49.098  3735: 3735 D/ActivityThread ]
setTargetHeapUtilization:0.75

[ 05-23 02:35:49.098  3735: 3735 D/ActivityThread ]
setTargetHeapMinFree:524288

[ 05-23 02:35:49.098  3735: 3735 V/ActivityThread ]
org.microg.nlp.backend.nominatim white listed for hwui

[ 05-23 02:35:49.863  3735: 3735 W/dalvikvm ]
VFY: unable to resolve exception class 2436 (Ljava/beans/IntrospectionException;)

[ 05-23 02:35:49.863  3735: 3735 W/dalvikvm ]
VFY: unable to find exception handler at addr 0x11

[ 05-23 02:35:49.863  3735: 3735 W/dalvikvm ]
VFY:  rejected Lorg/yaml/snakeyaml/representer/Representer$RepresentJavaBean;.representData (Ljava/lang/Object;)Lorg/yaml/snakeyaml/nodes/Node;

[ 05-23 02:35:49.863  3735: 3735 W/dalvikvm ]
VFY:  rejecting opcode 0x0d at 0x0011

[ 05-23 02:35:49.863  3735: 3735 W/dalvikvm ]
VFY:  rejected Lorg/yaml/snakeyaml/representer/Representer$RepresentJavaBean;.representData (Ljava/lang/Object;)Lorg/yaml/snakeyaml/nodes/Node;

[ 05-23 02:35:49.863  3735: 3735 W/dalvikvm ]
Verifier rejected class Lorg/yaml/snakeyaml/representer/Representer$RepresentJavaBean;

[ 05-23 02:35:49.863  3735: 3735 D/AndroidRuntime ]
Shutting down VM

[ 05-23 02:35:49.863  3735: 3735 W/dalvikvm ]
threadid=1: thread exiting with uncaught exception (group=0x415d7c20)

[ 05-23 02:35:49.863  3735: 3735 E/AndroidRuntime ]
FATAL EXCEPTION: main
Process: org.microg.nlp.backend.nominatim, PID: 3735
java.lang.VerifyError: org/yaml/snakeyaml/representer/Representer$RepresentJavaBean
	at org.yaml.snakeyaml.representer.Representer.<init>(Representer.java:44)
	at org.yaml.snakeyaml.Yaml.<init>(Yaml.java:63)
	at org.microg.address.Formatter.loadFile(Formatter.java:361)
	at org.microg.address.Formatter.readConfiguration(Formatter.java:332)
	at org.microg.address.Formatter.<init>(Formatter.java:61)
	at org.microg.address.Formatter.<init>(Formatter.java:56)
	at org.microg.nlp.backend.nominatim.BackendService.onCreate(BackendService.java:87)
	at android.app.ActivityThread.handleCreateService(ActivityThread.java:2596)
	at android.app.ActivityThread.access$1800(ActivityThread.java:144)
	at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1287)
	at android.os.Handler.dispatchMessage(Handler.java:102)
	at android.os.Looper.loop(Looper.java:136)
	at android.app.ActivityThread.main(ActivityThread.java:5179)
	at java.lang.reflect.Method.invokeNative(Native Method)
	at java.lang.reflect.Method.invoke(Method.java:515)
	at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:732)
	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:566)
	at dalvik.system.NativeStart.main(Native Method)

The title says it all… Anyway to test that backend ?

it was working fine. now about 1 month ago it does not give any results.
tested on 2 devices.

logcat will follow soon ...

If I understand correctly this code has became useless due to introduction of API keys. There seem to be no other free geocoding service. However, I can see some alternatives to make this service work again.

First, the reverse geocoding. It can be implemented with Natural Area Coding System. Neither database nor network is required simply transform coordinates into a NAC string. For me, for example, it would be enough since all I want is to use "Location" sorting (grouping actually) in my gallery. There are two simple features that can improve usability of such "procedural reverse geocoding":

• adjustable accuracy so a user can choose how long/accurate the generated strings are,
• a user maintained dictionary of places with user defined radii.

The real geocoding is more tricky as it requires database. If I can be an example again, I've got OsmAnd. It has its own offline maps and can do both geocoding and reverse geocoding at least inside the app. I am not sure if an API for this is exposed but I assume doing so wouldn't be too hard.

Of course the service would need to be renamed to OsmAndGeocoderBackend but that is rather trivial to do ;-)

Hello! I noticed that the location info returned (eg to gallery) are off by a couple of tens of kilometers. Is this a "blur" introduced for privacy reasons? I tried entering the coordinates into the online search form on the map quest website and the info shown there are exact.

Lookups on my smartphone fail with java.io.FileNotFoundException:

04-16 00:48:06.870  7673 14649 D NominatimGeocoderBackend: Requesting http://open.mapquestapi.com/nominatim/v1/reverse?format=json&accept-language=en&lat=39.106944&lon=-76.743056
04-16 00:48:06.981  7673 14649 W NominatimGeocoderBackend: java.io.FileNotFoundException: http://open.mapquestapi.com/nominatim/v1/reverse?format=json&accept-language=en&lat=39.106944&lon=-76.743056
04-16 00:48:06.981  7673 14649 W NominatimGeocoderBackend:  at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:238)
04-16 00:48:06.981  7673 14649 W NominatimGeocoderBackend:  at org.microg.nlp.backend.nominatim.BackendService$AsyncGetRequest.run(BackendService.java:189)
04-16 00:48:06.981  7673  7690 W NominatimGeocoderBackend: java.lang.NullPointerException: Attempt to get length of null array
04-16 00:48:06.981  7673  7690 W NominatimGeocoderBackend:  at java.lang.StringFactory.newStringFromBytes(StringFactory.java:45)
04-16 00:48:06.981  7673  7690 W NominatimGeocoderBackend:  at org.microg.nlp.backend.nominatim.BackendService$AsyncGetRequest.retrieveString(BackendService.java:220)
04-16 00:48:06.981  7673  7690 W NominatimGeocoderBackend:  at org.microg.nlp.backend.nominatim.BackendService.getFromLocation(BackendService.java:57)
04-16 00:48:06.981  7673  7690 W NominatimGeocoderBackend:  at org.microg.nlp.api.GeocoderBackendService$Backend.getFromLocation(GeocoderBackendService.java:72)
04-16 00:48:06.981  7673  7690 W NominatimGeocoderBackend:  at org.microg.nlp.api.GeocoderBackend$Stub.onTransact(GeocoderBackend.java:63)
04-16 00:48:06.981  7673  7690 W NominatimGeocoderBackend:  at android.os.Binder.execTransact(Binder.java:453)
04-16 00:48:06.983 13584 13606 E ReverseGeocoder: Fuuuuu
04-16 00:48:06.983 13584 13606 E ReverseGeocoder: java.io.IOException: no result
04-16 00:48:06.983 13584 13606 E ReverseGeocoder:   at android.location.Geocoder.getFromLocation(Geocoder.java:136)
...

If I curl the same URL, the response is The AppKey submitted with this request is invalid.. If the backend is not already sending an API key, than this is probably the problem.

Phone: Galaxy Nexus (maguro)
ROM: Unlegacy Android 4.4.4 (KTU84Q): http://builds.unlegacy-android.org/aosp-4.4/
microG Core 0.2.4-64 (fdroid)
Nominatim 1.2.1 (fdroid)

microG Self-Check reports everything working except geocoder. Trying to enable Nominatim in UnifiedNlp Settings is triggering the following crash:

I/ActivityManager( 472): Start proc org.microg.nlp.backend.nominatim for service org.microg.nlp.backend.nominatim/.BackendService: pid=2300 uid=10061 gids={50061, 3003}
D/dalvikvm( 2300): GC_CONCURRENT freed 162K, 7% free 3478K/3708K, paused 3ms+2ms, total 21ms
D/dalvikvm( 2300): GC_CONCURRENT freed 286K, 12% free 3601K/4060K, paused 2ms+1ms, total 15ms
D/dalvikvm( 2300): WAIT_FOR_CONCURRENT_GC blocked 11ms
D/dalvikvm( 2300): GC_CONCURRENT freed 193K, 7% free 3850K/4112K, paused 2ms+1ms, total 16ms
D/dalvikvm( 2300): GC_CONCURRENT freed 303K, 9% free 4019K/4392K, paused 0ms+2ms, total 16ms
W/dalvikvm( 2300): VFY: unable to resolve exception class 2436 (Ljava/beans/IntrospectionException;)
W/dalvikvm( 2300): VFY: unable to find exception handler at addr 0x11
W/dalvikvm( 2300): VFY: rejected Lorg/yaml/snakeyaml/representer/Representer$RepresentJavaBean;.representData (Ljava/lang/Object;)Lorg/yaml/snakeyaml/nodes/Node;
W/dalvikvm( 2300): VFY: rejecting opcode 0x0d at 0x0011
W/dalvikvm( 2300): VFY: rejected Lorg/yaml/snakeyaml/representer/Representer$RepresentJavaBean;.representData (Ljava/lang/Object;)Lorg/yaml/snakeyaml/nodes/Node;
W/dalvikvm( 2300): Verifier rejected class Lorg/yaml/snakeyaml/representer/Representer$RepresentJavaBean;
D/AndroidRuntime( 2300): Shutting down VM
W/dalvikvm( 2300): threadid=1: thread exiting with uncaught exception (group=0x41e1cc08)
E/AndroidRuntime( 2300): FATAL EXCEPTION: main
E/AndroidRuntime( 2300): Process: org.microg.nlp.backend.nominatim, PID: 2300
E/AndroidRuntime( 2300): java.lang.VerifyError: org/yaml/snakeyaml/representer/Representer$RepresentJavaBean
E/AndroidRuntime( 2300): at org.yaml.snakeyaml.representer.Representer.(Representer.java:44)
E/AndroidRuntime( 2300): at org.yaml.snakeyaml.Yaml.(Yaml.java:63)
E/AndroidRuntime( 2300): at org.microg.address.Formatter.loadFile(Formatter.java:361)
E/AndroidRuntime( 2300): at org.microg.address.Formatter.readConfiguration(Formatter.java:332)
E/AndroidRuntime( 2300): at org.microg.address.Formatter.(Formatter.java:61)
E/AndroidRuntime( 2300): at org.microg.address.Formatter.(Formatter.java:56)
E/AndroidRuntime( 2300): at org.microg.nlp.backend.nominatim.BackendService.onCreate(BackendService.java:87)
E/AndroidRuntime( 2300): at android.app.ActivityThread.handleCreateService(ActivityThread.java:2558)
E/AndroidRuntime( 2300): at android.app.ActivityThread.access$1800(ActivityThread.java:135)
E/AndroidRuntime( 2300): at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1278)
E/AndroidRuntime( 2300): at android.os.Handler.dispatchMessage(Handler.java:102)
E/AndroidRuntime( 2300): at android.os.Looper.loop(Looper.java:136)
E/AndroidRuntime( 2300): at android.app.ActivityThread.main(ActivityThread.java:5001)
E/AndroidRuntime( 2300): at java.lang.reflect.Method.invokeNative(Native Method)
E/AndroidRuntime( 2300): at java.lang.reflect.Method.invoke(Method.java:515)
E/AndroidRuntime( 2300): at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:795)
E/AndroidRuntime( 2300): at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:611)
E/AndroidRuntime( 2300): at dalvik.system.NativeStart.main(Native Method)
W/ActivityManager( 472): Timeout executing service: ServiceRecord{428c4688 u0 org.microg.nlp.backend.nominatim/.BackendService}
I/ActivityManager( 472): Crashing app skipping ANR: ProcessRecord{428c4d78 2300:org.microg.nlp.backend.nominatim/u0a61} Executing service org.microg.nlp.backend.nominatim/.BackendService
I/Process ( 2300): Sending signal. PID: 2300 SIG: 9
I/ActivityManager( 472): Process org.microg.nlp.backend.nominatim (pid 2300) has died.
D/ActivityManager( 472): cleanUpApplicationRecordLocked -- 2300
W/ActivityManager( 472): Scheduling restart of crashed service org.microg.nlp.backend.nominatim/.BackendService in 1000ms
W/InputMethodManagerService( 472): Window already focused, ignoring focus gain of: com.android.internal.view.IInputMethodClient$Stub$Proxy@42bd97a0 attribute=null, token = android.os.BinderProxy@42306270
V/Zygote ( 2315): Switching descriptor 32 to /dev/null
V/Zygote ( 2315): Switching descriptor 10 to /dev/null
I/ActivityManager( 472): Start proc org.microg.nlp.backend.nominatim for service org.microg.nlp.backend.nominatim/.BackendService: pid=2315 uid=10061 gids={50061, 3003}
D/dalvikvm( 2315): GC_CONCURRENT freed 155K, 6% free 3478K/3700K, paused 3ms+1ms, total 21ms
D/dalvikvm( 2315): GC_CONCURRENT freed 283K, 12% free 3585K/4036K, paused 2ms+1ms, total 15ms
D/dalvikvm( 2315): WAIT_FOR_CONCURRENT_GC blocked 13ms
D/dalvikvm( 2315): GC_CONCURRENT freed 198K, 7% free 3842K/4108K, paused 1ms+1ms, total 15ms
D/dalvikvm( 2315): GC_CONCURRENT freed 304K, 9% free 4003K/4376K, paused 1ms+2ms, total 14ms
W/dalvikvm( 2315): VFY: unable to resolve exception class 2436 (Ljava/beans/IntrospectionException;)
W/dalvikvm( 2315): VFY: unable to find exception handler at addr 0x11
W/dalvikvm( 2315): VFY: rejected Lorg/yaml/snakeyaml/representer/Representer$RepresentJavaBean;.representData (Ljava/lang/Object;)Lorg/yaml/snakeyaml/nodes/Node;
W/dalvikvm( 2315): VFY: rejecting opcode 0x0d at 0x0011
W/dalvikvm( 2315): VFY: rejected Lorg/yaml/snakeyaml/representer/Representer$RepresentJavaBean;.representData (Ljava/lang/Object;)Lorg/yaml/snakeyaml/nodes/Node;
W/dalvikvm( 2315): Verifier rejected class Lorg/yaml/snakeyaml/representer/Representer$RepresentJavaBean;
D/AndroidRuntime( 2315): Shutting down VM
W/dalvikvm( 2315): threadid=1: thread exiting with uncaught exception (group=0x41e1cc08)
E/AndroidRuntime( 2315): FATAL EXCEPTION: main
E/AndroidRuntime( 2315): Process: org.microg.nlp.backend.nominatim, PID: 2315
E/AndroidRuntime( 2315): java.lang.VerifyError: org/yaml/snakeyaml/representer/Representer$RepresentJavaBean
E/AndroidRuntime( 2315): at org.yaml.snakeyaml.representer.Representer.(Representer.java:44)
E/AndroidRuntime( 2315): at org.yaml.snakeyaml.Yaml.(Yaml.java:63)
E/AndroidRuntime( 2315): at org.microg.address.Formatter.loadFile(Formatter.java:361)
E/AndroidRuntime( 2315): at org.microg.address.Formatter.readConfiguration(Formatter.java:332)
E/AndroidRuntime( 2315): at org.microg.address.Formatter.(Formatter.java:61)
E/AndroidRuntime( 2315): at org.microg.address.Formatter.(Formatter.java:56)
E/AndroidRuntime( 2315): at org.microg.nlp.backend.nominatim.BackendService.onCreate(BackendService.java:87)
E/AndroidRuntime( 2315): at android.app.ActivityThread.handleCreateService(ActivityThread.java:2558)
E/AndroidRuntime( 2315): at android.app.ActivityThread.access$1800(ActivityThread.java:135)
E/AndroidRuntime( 2315): at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1278)
E/AndroidRuntime( 2315): at android.os.Handler.dispatchMessage(Handler.java:102)
E/AndroidRuntime( 2315): at android.os.Looper.loop(Looper.java:136)
E/AndroidRuntime( 2315): at android.app.ActivityThread.main(ActivityThread.java:5001)
E/AndroidRuntime( 2315): at java.lang.reflect.Method.invokeNative(Native Method)
E/AndroidRuntime( 2315): at java.lang.reflect.Method.invoke(Method.java:515)
E/AndroidRuntime( 2315): at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:795)
E/AndroidRuntime( 2315): at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:611)
E/AndroidRuntime( 2315): at dalvik.system.NativeStart.main(Native Method)
I/Process ( 2315): Sending signal. PID: 2315 SIG: 9
I/ActivityManager( 472): Process org.microg.nlp.backend.nominatim (pid 2315) has died.
D/ActivityManager( 472): cleanUpApplicationRecordLocked -- 2315
W/ActivityManager( 472): Service crashed 2 times, stopping: ServiceRecord{428c4688 u0 org.microg.nlp.backend.nominatim/.BackendService}

Please implement Reproducible Builds to allow using the developer signature on the default F-Droid server.

F-Droid supports reproducible builds of apps, so that anyone can run the build process again and reproduce the same APK as the original release. This means that F-Droid can verify that an app is 100% free software while still using the original developer’s APK signatures.

More info here.

1. src/main/AndroidManifest.xml
   Exported service does not require permission
Exported services (services which either set 'exported=true' or contain an intent-filter and do not specify 'exported=false') should define a permission that an entity must have in order to launch the service or bind to it. Without this, any application can use this service.

2. src/main/res/values/strings.xml
   The resource 'R.string.api_summary' appears to be unused
Unused resources make applications larger and slow down builds.
<string name="api_summary">MapQuest Developer API Key</string>

3. src/main/AndroidManifest.xml
   App is not indexable by Google Search; consider adding at least one Activity with an ACTION-VIEW intent filter. See issue explanation for more details.
Missing support for Firebase App Indexing
Adds URLs to get your app into the Google index, to get installs and traffic to your app from Google Search.