RuntimeException in Dexpler:Attempt to create VarNode of type byte about soot HOT 19 CLOSED

From the stack trace, it seems that the problematic jimple stmt has the following form: x = (C) b, where x's type is a subtype of RefLikeType and b's type is ByteType. Such typing is unexpected; it looks like the type assigner failing to correctly assign types to Local's in the Dexpler-generated jimple code.

I have experienced similar problem on a few apps. After closely looking at the problematic jimple code in my examples, I feel that fixing the typing problem is not straight-forward. One work around that I am planning to adopt in my case is to not run the type assigner at all, and write my analysis to work on mostly-untyped jimple code.

Could you print out the jimple code for the concerned method. You can do so by printing the stmt's in setCurrentMethod method of the soot.jimple.spark.builder.MethodNodeFactory class. The last method to be printed before the exception is thrown will be the interesting one.

from soot.

The method is: com.millennialmedia.android.PreCacheWorker: void run()

public void run()
{
    com.millennialmedia.android.PreCacheWorker $r0;
    java.lang.String[] $r1;
    java.lang.String $r2, $r19, $r21, $r22, $r24, $r26;
    java.lang.Exception $r3;
    java.lang.IllegalStateException $r4;
    org.apache.http.HttpEntity $r5;
    int $i0, $i1;
    byte $b2, $b4, $b5;
    java.lang.Throwable $r7, $r13;
    java.lang.Object[] $r8, $r25;
    boolean $z0, $z1, $z2;
    android.content.Context $r9;
    java.lang.InterruptedException $r10;
    java.lang.Class $r11;
    java.io.IOException $r12;
    com.millennialmedia.android.HttpGetRequest r14;
    com.millennialmedia.android.VideoAd r15;
    java.lang.Object r16;
    org.apache.http.HttpResponse $r17;
    long $l3;
    org.apache.http.Header $r20;
    java.io.InputStream $r23;

    $r0 := @this: com.millennialmedia.android.PreCacheWorker;
    entermonitor $r0;

 label0:
    $r1 = $r0.<com.millennialmedia.android.PreCacheWorker: java.lang.String[] cachedVideos>;
    $i1 = lengthof $r1;
    $i0 = 0;

 label1:
    if $i0 >= $i1 goto label22;

    $r2 = $r1[$i0];

 label2:
    r14 = new com.millennialmedia.android.HttpGetRequest;
    specialinvoke r14.<com.millennialmedia.android.HttpGetRequest: void <init>()>();
    $r17 = virtualinvoke r14.<com.millennialmedia.android.HttpGetRequest: org.apache.http.HttpResponse get(java.lang.String)>($r2);
    if $r17 != null goto label4;

    staticinvoke <com.millennialmedia.android.MMAdViewSDK$Log: void d(java.lang.String)>("Pre cache worker: HTTP response is null");

 label3:
    $i0 = $i0 + 1;
    goto label1;

 label4:
    $r5 = interfaceinvoke $r17.<org.apache.http.HttpResponse: org.apache.http.HttpEntity getEntity()>();

 label5:
    if $r5 != null goto label11;

 label6:
    staticinvoke <com.millennialmedia.android.MMAdViewSDK$Log: void d(java.lang.String)>("Pre cache worker: Null HTTP entity");

 label7:
    goto label3;

 label8:
    $r7 := @caughtexception;
    exitmonitor $r0;
    throw $r7;

 label9:
    $r3 := @caughtexception;

 label10:
    $b2 = 1;
    $r8 = newarray (java.lang.Object)[$b2];
    $r19 = virtualinvoke $r3.<java.lang.Exception: java.lang.String getMessage()>();
    $r8[0] = $r19;
    staticinvoke <com.millennialmedia.android.MMAdViewSDK$Log: void d(java.lang.String,java.lang.Object[])>("Pre cache worker HTTP error: %s", $r8);
    goto label3;

 label11:
    $l3 = interfaceinvoke $r5.<org.apache.http.HttpEntity: long getContentLength()>();
    $b4 = $l3 cmp 0L;
    if $b4 != 0 goto label12;

    staticinvoke <com.millennialmedia.android.MMAdViewSDK$Log: void d(java.lang.String)>("Pre cache worker: Millennial ad return failed. Zero content length returned.");
    goto label3;

 label12:
    $r20 = interfaceinvoke $r5.<org.apache.http.HttpEntity: org.apache.http.Header getContentType()>();
    if $r20 == null goto label3;

    $r21 = interfaceinvoke $r20.<org.apache.http.Header: java.lang.String getValue()>();
    if $r21 == null goto label3;

    $r22 = interfaceinvoke $r20.<org.apache.http.Header: java.lang.String getValue()>();
    $z0 = virtualinvoke $r22.<java.lang.String: boolean equalsIgnoreCase(java.lang.String)>("application/json");

 label13:
    if $z0 == 0 goto label3;

 label14:
    $r23 = interfaceinvoke $r5.<org.apache.http.HttpEntity: java.io.InputStream getContent()>();
    $r24 = staticinvoke <com.millennialmedia.android.HttpGetRequest: java.lang.String convertStreamToString(java.io.InputStream)>($r23);
    r15 = new com.millennialmedia.android.VideoAd;
    specialinvoke r15.<com.millennialmedia.android.VideoAd: void <init>(java.lang.String)>($r24);

 label15:
    if r15 == null goto label3;

 label16:
    $z1 = virtualinvoke r15.<com.millennialmedia.android.VideoAd: boolean isValid()>();

 label17:
    if $z1 == 0 goto label3;

    $b5 = 1;

 label18:
    r15.<com.millennialmedia.android.VideoAd: int downloadPriority> = $b5;
    $r9 = $r0.<com.millennialmedia.android.PreCacheWorker: android.content.Context appContext>;
    staticinvoke <com.millennialmedia.android.AdCache: void startDownloadTask(android.content.Context,java.lang.String,com.millennialmedia.android.CachedAd,com.millennialmedia.android.AdCache$AdCacheTaskListener)>($r9, null, r15, $r0);
    virtualinvoke $r0.<java.lang.Object: void wait()>();

 label19:
    goto label3;

 label20:
    $r10 := @caughtexception;

 label21:
    virtualinvoke $r10.<java.lang.InterruptedException: void printStackTrace()>();
    $r25 = newarray (java.lang.Object)[1];
    $r26 = virtualinvoke $r10.<java.lang.InterruptedException: java.lang.String getMessage()>();
    $r25[0] = $r26;
    staticinvoke <com.millennialmedia.android.MMAdViewSDK$Log: void e(java.lang.String,java.lang.Object[])>("Pre cache worker interrupted: %s", $r25);

 label22:
    $r11 = class "com/millennialmedia/android/PreCacheWorker";
    entermonitor $r11;

 label23:
    $z2 = 0;

 label24:
    <com.millennialmedia.android.PreCacheWorker: boolean busy> = $z2;
    exitmonitor $r11;

 label25:
    exitmonitor $r0;
    return;

 label26:
    $r4 := @caughtexception;

 label27:
    virtualinvoke $r4.<java.lang.IllegalStateException: void printStackTrace()>();
    staticinvoke <com.millennialmedia.android.MMAdViewSDK$Log: void d(java.lang.String)>("Pre cache worker: Millennial ad return failed. Invalid response data.");
    goto label3;

 label28:
    $r12 := @caughtexception;
    virtualinvoke $r12.<java.io.IOException: void printStackTrace()>();
    staticinvoke <com.millennialmedia.android.MMAdViewSDK$Log: void d(java.lang.String)>("Pre cache worker: Millennial ad return failed. Invalid response data.");

 label29:
    goto label3;

 label30:
    $r13 := @caughtexception;

 label31:
    r16 = (java.lang.Object) $b2;
    exitmonitor r16;

 label32:
    throw $r13;

    catch java.lang.Throwable from label0 to label2 with label8;
    catch java.lang.Exception from label2 to label5 with label9;
    catch java.lang.Throwable from label2 to label5 with label8;
    catch java.lang.Throwable from label6 to label7 with label8;
    catch java.lang.Throwable from label10 to label13 with label8;
    catch java.lang.IllegalStateException from label14 to label15 with label26;
    catch java.io.IOException from label14 to label15 with label28;
    catch java.lang.Throwable from label14 to label15 with label8;
    catch java.lang.Throwable from label16 to label17 with label8;
    catch java.lang.InterruptedException from label18 to label19 with label20;
    catch java.lang.Throwable from label18 to label19 with label8;
    catch java.lang.Throwable from label21 to label23 with label8;
    catch java.lang.Throwable from label24 to label25 with label30;
    catch java.lang.Throwable from label27 to label29 with label8;
    catch java.lang.Throwable from label31 to label32 with label30;
}

from soot.

From the jimple, it looks like the type assigner has no role to play in this problem. The stmt that is causing problem is, r16 = (java.lang.Object) $b2. And the the only definition of $b2 is, $b2 = 1. It looks like there is a problem in the creation of the jimple stmts.

If you want it to drill down into the problem, we may be able to get better insight if you can print out the smali code for the method. ("java -jar baksmali-1.3.2.jar app.apk" will generate smali code for each class of the app.apk in the out sub directory)

from soot.

Here's the full method in Smali, it's a bit lengthy, though:

.method public declared-synchronized run()V
    .registers 18
    .prologue
    .line 56
    monitor-enter p0
    :try_start_1
    move-object/from16 v0, p0
    iget-object v1, v0, Lcom/millennialmedia/android/PreCacheWorker;->cachedVideos:[Ljava/lang/String;
    .local v1, arr$:[Ljava/lang/String;
    array-length v11, v1
    .local v11, len$:I
    const/4 v9, 0x0
    .local v9, i$:I
    :goto_7
    if-ge v9, v11, :cond_a0
    aget-object v2, v1, v9
    :try_end_b
    .catchall {:try_start_1 .. :try_end_b} :catchall_2a
    .line 60
    .local v2, cachedVideo:Ljava/lang/String;
    :try_start_b
    new-instance v6, Lcom/millennialmedia/android/HttpGetRequest;
    invoke-direct {v6}, Lcom/millennialmedia/android/HttpGetRequest;->()V
    .line 61
    .local v6, httpGetRequest:Lcom/millennialmedia/android/HttpGetRequest;
    invoke-virtual {v6, v2}, Lcom/millennialmedia/android/HttpGetRequest;->get(Ljava/lang/String;)Lorg/apache/http/HttpResponse;
    move-result-object v8
    .line 62
    .local v8, httpResponse:Lorg/apache/http/HttpResponse;
    if-nez v8, :cond_1e
    .line 64
    const-string v13, "Pre cache worker: HTTP response is null"
    invoke-static {v13}, Lcom/millennialmedia/android/MMAdViewSDK$Log;->d(Ljava/lang/String;)V
    .line 56
    .end local v6           #httpGetRequest:Lcom/millennialmedia/android/HttpGetRequest;
    .end local v8           #httpResponse:Lorg/apache/http/HttpResponse;
    :cond_1b
    :goto_1b
    add-int/lit8 v9, v9, 0x1
    goto :goto_7
    .line 67
    .restart local v6       #httpGetRequest:Lcom/millennialmedia/android/HttpGetRequest;
    .restart local v8       #httpResponse:Lorg/apache/http/HttpResponse;
    :cond_1e
    invoke-interface {v8}, Lorg/apache/http/HttpResponse;->getEntity()Lorg/apache/http/HttpEntity;
    :try_end_21
    .catchall {:try_start_b .. :try_end_21} :catchall_2a
    .catch Ljava/lang/Exception; {:try_start_b .. :try_end_21} :catch_2d
    move-result-object v5
    .line 75
    .local v5, httpEntity:Lorg/apache/http/HttpEntity;
    if-nez v5, :cond_3e
    .line 77
    :try_start_24
    const-string v13, "Pre cache worker: Null HTTP entity"
    invoke-static {v13}, Lcom/millennialmedia/android/MMAdViewSDK$Log;->d(Ljava/lang/String;)V
    :try_end_29
    .catchall {:try_start_24 .. :try_end_29} :catchall_2a
    goto :goto_1b
    .line 56
    .end local v1           #arr$:[Ljava/lang/String;
    .end local v2           #cachedVideo:Ljava/lang/String;
    .end local v5           #httpEntity:Lorg/apache/http/HttpEntity;
    .end local v6           #httpGetRequest:Lcom/millennialmedia/android/HttpGetRequest;
    .end local v8           #httpResponse:Lorg/apache/http/HttpResponse;
    .end local v9           #i$:I
    .end local v11           #len$:I
    :catchall_2a
    move-exception v13
    monitor-exit p0
    throw v13
    .line 69
    .restart local v1       #arr$:[Ljava/lang/String;
    .restart local v2       #cachedVideo:Ljava/lang/String;
    .restart local v9       #i$:I
    .restart local v11       #len$:I
    :catch_2d
    move-exception v3
    .line 71
    .local v3, e:Ljava/lang/Exception;
    :try_start_2e
    const-string v13, "Pre cache worker HTTP error: %s"
    const/4 v14, 0x1
    new-array v14, v14, [Ljava/lang/Object;
    const/4 v15, 0x0
    invoke-virtual {v3}, Ljava/lang/Exception;->getMessage()Ljava/lang/String;
    move-result-object v16
    aput-object v16, v14, v15
    invoke-static {v13, v14}, Lcom/millennialmedia/android/MMAdViewSDK$Log;->d(Ljava/lang/String;[Ljava/lang/Object;)V
    goto :goto_1b
    .line 81
    .end local v3           #e:Ljava/lang/Exception;
    .restart local v5       #httpEntity:Lorg/apache/http/HttpEntity;
    .restart local v6       #httpGetRequest:Lcom/millennialmedia/android/HttpGetRequest;
    .restart local v8       #httpResponse:Lorg/apache/http/HttpResponse;
    :cond_3e
    invoke-interface {v5}, Lorg/apache/http/HttpEntity;->getContentLength()J
    move-result-wide v13
    const-wide/16 v15, 0x0
    cmp-long v13, v13, v15
    if-nez v13, :cond_4e
    .line 83
    const-string v13, "Pre cache worker: Millennial ad return failed. Zero content length returned."
    invoke-static {v13}, Lcom/millennialmedia/android/MMAdViewSDK$Log;->d(Ljava/lang/String;)V
    goto :goto_1b
    .line 87
    :cond_4e
    invoke-interface {v5}, Lorg/apache/http/HttpEntity;->getContentType()Lorg/apache/http/Header;
    move-result-object v7
    .line 88
    .local v7, httpHeader:Lorg/apache/http/Header;
    if-eqz v7, :cond_1b
    .line 90
    invoke-interface {v7}, Lorg/apache/http/Header;->getValue()Ljava/lang/String;
    move-result-object v13
    if-eqz v13, :cond_1b
    .line 92
    invoke-interface {v7}, Lorg/apache/http/Header;->getValue()Ljava/lang/String;
    move-result-object v13
    const-string v14, "application/json"
    invoke-virtual {v13, v14}, Ljava/lang/String;->equalsIgnoreCase(Ljava/lang/String;)Z
    :try_end_63
    .catchall {:try_start_2e .. :try_end_63} :catchall_2a
    move-result v13
    if-eqz v13, :cond_1b
    .line 94
    const/4 v12, 0x0
    .line 99
    .local v12, videoAd:Lcom/millennialmedia/android/VideoAd;
    :try_start_67
    invoke-interface {v5}, Lorg/apache/http/HttpEntity;->getContent()Ljava/io/InputStream;
    move-result-object v13
    invoke-static {v13}, Lcom/millennialmedia/android/HttpGetRequest;->convertStreamToString(Ljava/io/InputStream;)Ljava/lang/String;
    move-result-object v10
    .line 100
    .local v10, json:Ljava/lang/String;
    new-instance v12, Lcom/millennialmedia/android/VideoAd;
    .end local v12           #videoAd:Lcom/millennialmedia/android/VideoAd;
    invoke-direct {v12, v10}, Lcom/millennialmedia/android/VideoAd;->(Ljava/lang/String;)V
    :try_end_74
    .catchall {:try_start_67 .. :try_end_74} :catchall_2a
    .catch Ljava/lang/IllegalStateException; {:try_start_67 .. :try_end_74} :catch_a9
    .catch Ljava/io/IOException; {:try_start_67 .. :try_end_74} :catch_b4
    .line 115
    .restart local v12       #videoAd:Lcom/millennialmedia/android/VideoAd;
    if-eqz v12, :cond_1b
    :try_start_76
    invoke-virtual {v12}, Lcom/millennialmedia/android/VideoAd;->isValid()Z
    :try_end_79
    .catchall {:try_start_76 .. :try_end_79} :catchall_2a
    move-result v13
    if-eqz v13, :cond_1b
    .line 119
    const/4 v13, 0x1
    :try_start_7d
    iput v13, v12, Lcom/millennialmedia/android/VideoAd;->downloadPriority:I
    .line 120
    move-object/from16 v0, p0
    iget-object v13, v0, Lcom/millennialmedia/android/PreCacheWorker;->appContext:Landroid/content/Context;
    const/4 v14, 0x0
    move-object/from16 v0, p0
    invoke-static {v13, v14, v12, v0}, Lcom/millennialmedia/android/AdCache;->startDownloadTask(Landroid/content/Context;Ljava/lang/String;Lcom/millennialmedia/android/CachedAd;Lcom/millennialmedia/android/AdCache$AdCacheTaskListener;)V
    .line 121
    invoke-virtual/range {p0 .. p0}, Ljava/lang/Object;->wait()V
    :try_end_8c
    .catchall {:try_start_7d .. :try_end_8c} :catchall_2a
    .catch Ljava/lang/InterruptedException; {:try_start_7d .. :try_end_8c} :catch_8d
    goto :goto_1b
    .line 123
    :catch_8d
    move-exception v3
    .line 125
    .local v3, e:Ljava/lang/InterruptedException;
    :try_start_8e
    invoke-virtual {v3}, Ljava/lang/InterruptedException;->printStackTrace()V
    .line 126
    const-string v13, "Pre cache worker interrupted: %s"
    const/4 v14, 0x1
    new-array v14, v14, [Ljava/lang/Object;
    const/4 v15, 0x0
    invoke-virtual {v3}, Ljava/lang/InterruptedException;->getMessage()Ljava/lang/String;
    move-result-object v16
    aput-object v16, v14, v15
    invoke-static {v13, v14}, Lcom/millennialmedia/android/MMAdViewSDK$Log;->e(Ljava/lang/String;[Ljava/lang/Object;)V
    .line 134
    .end local v2           #cachedVideo:Ljava/lang/String;
    .end local v3           #e:Ljava/lang/InterruptedException;
    .end local v5           #httpEntity:Lorg/apache/http/HttpEntity;
    .end local v6           #httpGetRequest:Lcom/millennialmedia/android/HttpGetRequest;
    .end local v7           #httpHeader:Lorg/apache/http/Header;
    .end local v8           #httpResponse:Lorg/apache/http/HttpResponse;
    .end local v10           #json:Ljava/lang/String;
    .end local v12           #videoAd:Lcom/millennialmedia/android/VideoAd;
    :cond_a0
    const-class v14, Lcom/millennialmedia/android/PreCacheWorker;
    monitor-enter v14
    :try_end_a3
    .catchall {:try_start_8e .. :try_end_a3} :catchall_2a
    .line 136
    const/4 v13, 0x0
    :try_start_a4
    sput-boolean v13, Lcom/millennialmedia/android/PreCacheWorker;->busy:Z
    .line 137
    monitor-exit v14
    :try_end_a7
    .catchall {:try_start_a4 .. :try_end_a7} :catchall_bf
    .line 138
    monitor-exit p0
    return-void
    .line 102
    .restart local v2       #cachedVideo:Ljava/lang/String;
    .restart local v5       #httpEntity:Lorg/apache/http/HttpEntity;
    .restart local v6       #httpGetRequest:Lcom/millennialmedia/android/HttpGetRequest;
    .restart local v7       #httpHeader:Lorg/apache/http/Header;
    .restart local v8       #httpResponse:Lorg/apache/http/HttpResponse;
    :catch_a9
    move-exception v4
    .line 104
    .local v4, e1:Ljava/lang/IllegalStateException;
    :try_start_aa
    invoke-virtual {v4}, Ljava/lang/IllegalStateException;->printStackTrace()V
    .line 105
    const-string v13, "Pre cache worker: Millennial ad return failed. Invalid response data."
    invoke-static {v13}, Lcom/millennialmedia/android/MMAdViewSDK$Log;->d(Ljava/lang/String;)V
    goto/16 :goto_1b
    .line 108
    .end local v4           #e1:Ljava/lang/IllegalStateException;
    :catch_b4
    move-exception v4
    .line 110
    .local v4, e1:Ljava/io/IOException;
    invoke-virtual {v4}, Ljava/io/IOException;->printStackTrace()V
    .line 111
    const-string v13, "Pre cache worker: Millennial ad return failed. Invalid response data."
    invoke-static {v13}, Lcom/millennialmedia/android/MMAdViewSDK$Log;->d(Ljava/lang/String;)V
    :try_end_bd
    .catchall {:try_start_aa .. :try_end_bd} :catchall_2a
    goto/16 :goto_1b
    .line 137
    .end local v2           #cachedVideo:Ljava/lang/String;
    .end local v4           #e1:Ljava/io/IOException;
    .end local v5           #httpEntity:Lorg/apache/http/HttpEntity;
    .end local v6           #httpGetRequest:Lcom/millennialmedia/android/HttpGetRequest;
    .end local v7           #httpHeader:Lorg/apache/http/Header;
    .end local v8           #httpResponse:Lorg/apache/http/HttpResponse;
    :catchall_bf
    move-exception v13
    :try_start_c0
    monitor-exit v14
    :try_end_c1
    .catchall {:try_start_c0 .. :try_end_c1} :catchall_bf
    :try_start_c1
    throw v13
    :try_end_c2
    .catchall {:try_start_c1 .. :try_end_c2} :catchall_2a
.end method

from soot.

The relevant piece of .smali code seems to be:

const/4 v14, 0x1
new-array v14, v14, [Ljava/lang/Object;

The register v14 is first used to store a constant int value "1", and then it is used to store an array.

The corresponding jimple code is:
$b2 = 1;
$r8 = newarray (java.lang.Object)[$b2];

The LocalSplitter correctly introduces two different locals $b2 and $r8 (one for each DU web). However, it for some reason, does not use "$r8" in place of "$b2" in the following cast stmt. "$b2" should not be the operand in the cast stmt because in the smali code, "v14 = 0x1" definition even does not reach the corresponding cast instruction.

r16 = (java.lang.Object) $b2;

In summary, I dont yet have a solution here. The LocalSplitter does not appear to be doing its job right.

from soot.

Could you try adding the following code right after the call to splitLocals() in DexBody class (line ~475), and rerun Dexpler.

soot.jimple.toolkits.base.Aggregator.v().transform(jBody);

from soot.

Interestingly, the output of splitLocals() seems to be fine. The method correctly generates different locals for the different uses of r14 and I did not find any type mismatches in the output. Next, I'll try to find out what happens afterwards - something seems to go wrong on the way from u14 to $b2.

Method body at the end of splitLocals():

public void run()
{
    unknown $u17, $u0, $u1, $u2, $u3, $u4, $u5, $u6, $u7, $u8, $u9, $u10, $u11, $u12, $u13, $u14, $u15, $u16, $u-1, $u-1#2, $u13#2, $u13#3, $u13#4, $u14#2, $u-1#3, $u-1#4, $u13#5, $u15#2, $u13#6, $u13#7, $u-1#5, $u-1#6, $u13#8, $u-1#7, $u13#9, $u14#3, $u-1#8, $u13#10, $u-1#9, $u13#11, $u-1#10, $u-1#11, $u13#12, $u13#13, $u0#2, $u13#14, $u14#4, $u0#3, $u3#2, $u13#15, $u14#5, $u14#6, $u15#3, $u-1#12, $u16#2, $u14#7, $u13#16, $u13#17, $u4#2, $u13#18, $u13#19;

    $u17 := @this: com.millennialmedia.android.PreCacheWorker;
    entermonitor $u17;

 label0:
    $u0 = $u17;
    $u1 = $u0.<com.millennialmedia.android.PreCacheWorker: java.lang.String[] cachedVideos>;
    $u11 = lengthof $u1;
    $u9 = 0;

 label1:
    if $u9 >= $u11 goto label22;

    $u2 = $u1[$u9];

 label2:
    $u6 = new com.millennialmedia.android.HttpGetRequest;
    specialinvoke $u6.<com.millennialmedia.android.HttpGetRequest: void <init>()>();
    $u-1 = virtualinvoke $u6.<com.millennialmedia.android.HttpGetRequest: org.apache.http.HttpResponse get(java.lang.String)>($u2);
    $u8 = $u-1;
    if $u8 != 0 goto label4;

    $u13 = "Pre cache worker: HTTP response is null";
    staticinvoke <com.millennialmedia.android.MMAdViewSDK$Log: void d(java.lang.String)>($u13);

 label3:
    $u9 = $u9 + 1;
    goto label1;

 label4:
    $u-1#2 = interfaceinvoke $u8.<org.apache.http.HttpResponse: org.apache.http.HttpEntity getEntity()>();

 label5:
    $u5 = $u-1#2;
    if $u5 != 0 goto label11;

 label6:
    $u13#2 = "Pre cache worker: Null HTTP entity";
    staticinvoke <com.millennialmedia.android.MMAdViewSDK$Log: void d(java.lang.String)>($u13#2);

 label7:
    goto label3;

 label8:
    $u13#3 := @caughtexception;
    exitmonitor $u17;
    throw $u13#3;

 label9:
    $u3 := @caughtexception;

 label10:
    $u13#4 = "Pre cache worker HTTP error: %s";
    $u14 = 1;
    $u14#2 = newarray (java.lang.Object)[$u14];
    $u15 = 0;
    $u-1#3 = virtualinvoke $u3.<java.lang.Exception: java.lang.String getMessage()>();
    $u16 = $u-1#3;
    $u14#2[$u15] = $u16;
    staticinvoke <com.millennialmedia.android.MMAdViewSDK$Log: void d(java.lang.String,java.lang.Object[])>($u13#4, $u14#2);
    goto label3;

 label11:
    $u-1#4 = interfaceinvoke $u5.<org.apache.http.HttpEntity: long getContentLength()>();
    $u13#5 = $u-1#4;
    $u15#2 = 0L;
    $u13#6 = $u13#5 cmp $u15#2;
    if $u13#6 != 0 goto label12;

    $u13#7 = "Pre cache worker: Millennial ad return failed. Zero content length returned.";
    staticinvoke <com.millennialmedia.android.MMAdViewSDK$Log: void d(java.lang.String)>($u13#7);
    goto label3;

 label12:
    $u-1#5 = interfaceinvoke $u5.<org.apache.http.HttpEntity: org.apache.http.Header getContentType()>();
    $u7 = $u-1#5;
    if $u7 == 0 goto label3;

    $u-1#6 = interfaceinvoke $u7.<org.apache.http.Header: java.lang.String getValue()>();
    $u13#8 = $u-1#6;
    if $u13#8 == 0 goto label3;

    $u-1#7 = interfaceinvoke $u7.<org.apache.http.Header: java.lang.String getValue()>();
    $u13#9 = $u-1#7;
    $u14#3 = "application/json";
    $u-1#8 = virtualinvoke $u13#9.<java.lang.String: boolean equalsIgnoreCase(java.lang.String)>($u14#3);

 label13:
    $u13#10 = $u-1#8;
    if $u13#10 == 0 goto label3;

 label14:
    $u-1#9 = interfaceinvoke $u5.<org.apache.http.HttpEntity: java.io.InputStream getContent()>();
    $u13#11 = $u-1#9;
    $u-1#10 = staticinvoke <com.millennialmedia.android.HttpGetRequest: java.lang.String convertStreamToString(java.io.InputStream)>($u13#11);
    $u10 = $u-1#10;
    $u12 = new com.millennialmedia.android.VideoAd;
    specialinvoke $u12.<com.millennialmedia.android.VideoAd: void <init>(java.lang.String)>($u10);

 label15:
    if $u12 == 0 goto label3;

 label16:
    $u-1#11 = virtualinvoke $u12.<com.millennialmedia.android.VideoAd: boolean isValid()>();

 label17:
    $u13#12 = $u-1#11;
    if $u13#12 == 0 goto label3;

    $u13#13 = 1;

 label18:
    $u12.<com.millennialmedia.android.VideoAd: int downloadPriority> = $u13#13;
    $u0#2 = $u17;
    $u13#14 = $u0#2.<com.millennialmedia.android.PreCacheWorker: android.content.Context appContext>;
    $u14#4 = 0;
    $u0#3 = $u17;
    staticinvoke <com.millennialmedia.android.AdCache: void startDownloadTask(android.content.Context,java.lang.String,com.millennialmedia.android.CachedAd,com.millennialmedia.android.AdCache$AdCacheTaskListener)>($u13#14, $u14#4, $u12, $u0#3);
    virtualinvoke $u17.<java.lang.Object: void wait()>();

 label19:
    goto label3;

 label20:
    $u3#2 := @caughtexception;

 label21:
    virtualinvoke $u3#2.<java.lang.InterruptedException: void printStackTrace()>();
    $u13#15 = "Pre cache worker interrupted: %s";
    $u14#5 = 1;
    $u14#6 = newarray (java.lang.Object)[$u14#5];
    $u15#3 = 0;
    $u-1#12 = virtualinvoke $u3#2.<java.lang.InterruptedException: java.lang.String getMessage()>();
    $u16#2 = $u-1#12;
    $u14#6[$u15#3] = $u16#2;
    staticinvoke <com.millennialmedia.android.MMAdViewSDK$Log: void e(java.lang.String,java.lang.Object[])>($u13#15, $u14#6);

 label22:
    $u14#7 = class "com/millennialmedia/android/PreCacheWorker";
    entermonitor $u14#7;

 label23:
    $u13#16 = 0;

 label24:
    <com.millennialmedia.android.PreCacheWorker: boolean busy> = $u13#16;
    exitmonitor $u14#7;

 label25:
    exitmonitor $u17;
    return;

 label26:
    $u4 := @caughtexception;

 label27:
    virtualinvoke $u4.<java.lang.IllegalStateException: void printStackTrace()>();
    $u13#17 = "Pre cache worker: Millennial ad return failed. Invalid response data.";
    staticinvoke <com.millennialmedia.android.MMAdViewSDK$Log: void d(java.lang.String)>($u13#17);
    goto label3;

 label28:
    $u4#2 := @caughtexception;
    virtualinvoke $u4#2.<java.io.IOException: void printStackTrace()>();
    $u13#18 = "Pre cache worker: Millennial ad return failed. Invalid response data.";
    staticinvoke <com.millennialmedia.android.MMAdViewSDK$Log: void d(java.lang.String)>($u13#18);

 label29:
    goto label3;

 label30:
    $u13#19 := @caughtexception;

 label31:
    exitmonitor $ #7;

 label32:
    throw $u13#19;

    catch java.lang.Throwable from label0 to label2 with label8;
    catch java.lang.Exception from label2 to label5 with label9;
    catch java.lang.Throwable from label2 to label5 with label8;
    catch java.lang.Throwable from label6 to label7 with label8;
    catch java.lang.Throwable from label10 to label13 with label8;
    catch java.lang.IllegalStateException from label14 to label15 with label26;
    catch java.io.IOException from label14 to label15 with label28;
    catch java.lang.Throwable from label14 to label15 with label8;
    catch java.lang.Throwable from label16 to label17 with label8;
    catch java.lang.InterruptedException from label18 to label19 with label20;
    catch java.lang.Throwable from label18 to label19 with label8;
    catch java.lang.Throwable from label21 to label23 with label8;
    catch java.lang.Throwable from label24 to label25 with label30;
    catch java.lang.Throwable from label27 to label29 with label8;
    catch java.lang.Throwable from label31 to label32 with label30;
}

from soot.

In the above (i.e., body after splitLocals), I do not see the problematic cast stmt in the block labeled "label31". I wonder why.

from soot.

I guess it's not there because the locals are not typed yet, so if everything is of the same type "unknown" there is no need to cast anything.

from soot.

It is interesting! Actually both Dalvik and Java bytecodes have opcodes for checking cast. Some even in the untyped jimple, some cast stmts, which correspond to the checkcast opcodes, will appear. However, you are also right that some cast stmts (as in this example) are inserted by some component (probably the type assigner) of Soot to satisfy typing constraints.

Btw, did you try the suggestion of using Aggregator after splitLocals()?

from soot.

I tried to insert the Aggregator call and it seems to resolve the problem at hand. However, I am by no means sure that this fix works in the general case. It should never break anything, but if there is a case not handled by the aggregator (e.g. multiple uses of the same definition), we might still run into the same issue.

Anyway, I'll check this in after it has received some further testing. I am currently investigating a follow-up issue which breaks the SPARK callgraph builder on the generated Jimple methods. I just want to make sure that this is not due to some weird effect introduced by the aggregator before I push it to GIT.

from soot.

You can send me this app as well.
Thanks!
/Alex

On Wed, 2013-05-08 at 04:03 -0700, Steven Arzt wrote:

In a test application I can provide upon request
(v2_com.zeptolab.ctr.ads_1_20_Cut the Rope FULL FREE.apk), the
following exception occurs in Dexpler:

Exception in thread "main" java.lang.RuntimeException: Attempt to
create VarNode of type byte
at soot.jimple.spark.pag.VarNode.(VarNode.java:106)
at soot.jimple.spark.pag.LocalVarNode.(LocalVarNode.java:44)
at soot.jimple.spark.pag.PAG.makeLocalVarNode(PAG.java:540)
at
soot.jimple.spark.builder.MethodNodeFactory.caseLocal(MethodNodeFactory.java:207)
at soot.jimple.internal.JimpleLocal.apply(JimpleLocal.java:132)
at
soot.jimple.spark.builder.MethodNodeFactory.caseCastExpr(MethodNodeFactory.java:184)
at
soot.jimple.internal.AbstractCastExpr.apply(AbstractCastExpr.java:127)
at soot.jimple.spark.builder.MethodNodeFactory
$1.caseAssignStmt(MethodNodeFactory.java:74)
at soot.jimple.internal.JAssignStmt.apply(JAssignStmt.java:221)
at
soot.jimple.spark.builder.MethodNodeFactory.handleStmt(MethodNodeFactory.java:67)
at soot.jimple.spark.pag.MethodPAG.buildNormal(MethodPAG.java:181)
at soot.jimple.spark.pag.MethodPAG.build(MethodPAG.java:147)
at
soot.jimple.spark.solver.OnFlyCallGraph.processReachables(OnFlyCallGraph.java:64)
at
soot.jimple.spark.solver.OnFlyCallGraph.build(OnFlyCallGraph.java:56)
at
soot.jimple.spark.solver.PropWorklist.handleVarNode(PropWorklist.java:123)
at
soot.jimple.spark.solver.PropWorklist.propagate(PropWorklist.java:53)
at
soot.jimple.spark.SparkTransformer.internalTransform(SparkTransformer.java:152)
at soot.SceneTransformer.transform(SceneTransformer.java:39)
at soot.Transform.apply(Transform.java:89)
at soot.RadioScenePack.internalApply(RadioScenePack.java:57)
at
soot.jimple.toolkits.callgraph.CallGraphPack.internalApply(CallGraphPack.java:49)
at soot.Pack.apply(Pack.java:114)
at soot.PackManager.runWholeProgramPacks(PackManager.java:464)
at soot.PackManager.runPacksNormally(PackManager.java:373)
at soot.PackManager.runPacks(PackManager.java:335)
at soot.jimple.infoflow.Infoflow.computeInfoflow(Infoflow.java:226)
at
soot.jimple.infoflow.android.SetupApplication.runInfoflow(SetupApplication.java:245)
at soot.jimple.infoflow.android.TestApps.Test.main(Test.java:133)

—
Reply to this email directly or view it on GitHub.

from soot.

ok, inserting a call to the aggregator does not fix the issue. This just makes the case a bit harder for the DexNullTransformer running afterwards since it now needs to cope with inlined zero values. After extending the DexNullTransformer and getting this side effect out of the way, I end up with the same error as before, SPARK again fails with "Attempt to create VarNode of type byte". I will thus not check in the aggregator call. I will however check in the extensions to the DexNullTransformer as they are a good idea anyway - they enlarge the scope of what the null transformer can handle correctly.

@Alexandre-Bartel I will collect the failing APKs and send them to you next week

from soot.

I think I found the issue. The LocalSplitter only works correctly if there is no unreachable code, therefore the UnreachableCodeEliminator is invoked first. However, the two components use two different configurations for the CFG:

LocalSplitter: ExceptionalUnitGraph graph = new ExceptionalUnitGraph(body,throwAnalysis,true);
UnreachableCodeEliminator: stmtGraph = new ExceptionalUnitGraph(body, PedanticThrowAnalysis.v(), false);

From the UnreachableCodeEliminator's view, the exception handler at label30 is reachable. For the LocalSplitter, there is however no statement that has the handler as a successor, so splitting fails as expected. Making the splitter more conservative by passing "false" for the last parameter could be an option - if that doesn't break anything else.

from soot.

A good catch! I agree that the same throw analysis (i.e., supposedly DalvikThrowAnalysis) should be used in all cases.

But in this particular case, it looks like (in your above print-out of the jimple body after the local splitter is run) the jimple stmt "exitmonitor $u14#7;" in the block labeled label24 can throw an exception that is caught by the handler at label30. So label30 is reachable. (label24 is reachable on the path label1 -> label22 -> label23 -> label 24).

I checked lines 399-400 in UnitThrowAnalysis, which is super class of DalvikThrowAnalysis, do suggest that exitmonitor statement can throw two kinds of exceptions.

from soot.

One thing that looks suspicious in your above print-out of the jimple body after the local splitter is run is the exitmonitor stmt shown below. This is the exitmonitor statement that consumes the result of the problematic casting operation (i.e., (java.lang.Object) $b2). Notice that the local in the exitmonitor stmt is "$ #7" --- there is a space in the local name and I dont see that local in the list of locals at the method head! I wonder whether it is a problem with github's formatting or it is a hint of the cause of our problem. I expected the local operand of the exitmonitor stmt to be $u14#2 that is defined in label10 block based on the print-out of the typed jimple code.

label31:
exitmonitor $ #7;

from soot.

@saswatanand The issue is that two components had different definitions of what was "dead" code. In this case, label30 is reachable - but then, both components must agree on that. If something is not removed during dead code elimination, then the LocalSplitter must find a combination of definition and use - if not, the respective code remains in the default split. If the local is however of multiple different types in the default split because of such oversight, the type assigner breaks afterwards.

Callgraph generation now works pretty well on this APK, so I'll just run a few regression tests with an APK test suite I have anyway (DroidBench). If nothing ugly turns up there, I'll push that code and close the issue.

from soot.

so label30 is not reachable as per the pedantic exception analysis? From the name of it, I was expecting that pedantic exception analysis would be very super conservative.

Great job hunting down this bug! After you push, I will try it on an apk on which the type assigner is currently throwing a runtime exception.

from soot.

No, the other way round: label30 was reachable for the UnreachableCodeEliminator using the pedantic analysis, so it did not get removed. However, the LocalSplitter was more coarse-grained and thus did not find a path to it, so it got ignored there.

The code is online, for my test cases it did not break anything and the APK now works well.

from soot.