mobile-insight / mobileinsight-core Goto Github PK
View Code? Open in Web Editor NEWMobile Network Intelligence Made Easy -- MobileInsight core functionalities
Home Page: http://www.mobileinsight.net
License: Other
Mobile Network Intelligence Made Easy -- MobileInsight core functionalities
Home Page: http://www.mobileinsight.net
License: Other
RSSI value is missing in LtePhySubpktFmt_v1_Scmr_v36 but when I processed the same packet with LtePhySubpktFmt_v1_Scmr_v35, I got the same values along with RSSI value. If you guys have the exact format fot v36, please update dm-collector-c. Thanks.
What are you trying to get done: I collected some traces using the mobile version of Mobile Insight. These traces are then copied to my laptop so that i can run Mobile Insight-core. I made a script which reads the raw data from the traces and saves the contents into a csv file.
what has happened / what went wrong: My guess is my script can't read the *.db files generated by mobile Insight (mi-gui). i am also thinking that the root of my problem is related to the LteNasAnalyzer() function.
and what did you expect? I was hoping to have some csv files generated.
# -*- coding: utf-8 -*
#!/usr/bin/python
import os
import sys
import pandas as pd
import numpy as np
from sklearn.svm import SVC
from itertools import chain
from sklearn.model_selection import cross_val_score
from sklearn.ensemble import RandomForestClassifier
from sklearn import tree
import lightgbm as lgb
from sklearn import metrics
import matplotlib.pyplot as plt
support_messages = {'LTE_MAC_Configuration':1, 'LTE_MAC_DL_Transport_Block':2, 'LTE_MAC_Rach_Attempt':3, 'LTE_MAC_Rach_Trigger':4, 'LTE_MAC_UL_Buffer_Status_Internal':5, 'LTE_MAC_UL_Transport_Block':7, 'LTE_MAC_UL_Tx_Statistics':8, 'LTE_NAS_EMM_OTA_Incoming_Packet':11, 'LTE_NAS_EMM_OTA_Outgoing_Packet':12, 'LTE_NAS_EMM_State':13, 'LTE_NAS_ESM_OTA_Incoming_Packet':14, 'LTE_NAS_ESM_OTA_Outgoing_Packet':15, 'LTE_NAS_ESM_State':16, 'LTE_PDCP_DL_Cipher_Data_PDU':17, 'LTE_PDCP_DL_Config':18, 'LTE_PDCP_DL_Ctrl_PDU':19, 'LTE_PDCP_DL_SRB_Integrity_Data_PDU':20, 'LTE_PDCP_DL_Stats':21, 'LTE_PDCP_UL_Cipher_Data_PDU':22, 'LTE_PDCP_UL_Config':23, 'LTE_PDCP_UL_Ctrl_PDU':24, 'LTE_PDCP_UL_Data_PDU':25, 'LTE_PDCP_UL_SRB_Integrity_Data_PDU':26, 'LTE_PDCP_UL_Stats':27, 'LTE_PHY_BPLMN_Cell_Confirm':28, 'LTE_PHY_BPLMN_Cell_Request':29, 'LTE_PHY_Connected_Mode_Intra_Freq_Meas':30, 'LTE_PHY_Connected_Mode_Neighbor_Measurement':31, 'LTE_PHY_Idle_Neighbor_Cell_Meas':32, 'LTE_PHY_Inter_RAT_CDMA_Measurement':33, 'LTE_PHY_Inter_RAT_Measurement':34, 'LTE_PHY_PDCCH_Decoding_Result':35, 'LTE_PHY_PDCCH_PHICH_Indication_Report':36, 'LTE_PHY_PDSCH_Decoding_Result':37, 'LTE_PHY_PDSCH_Packet':38, 'LTE_PHY_PDSCH_Stat_Indication':39, 'LTE_PHY_PUCCH_CSF':40, 'LTE_PHY_PUCCH_Power_Control':41, 'LTE_PHY_PUCCH_Tx_Report':42, 'LTE_PHY_PUSCH_CSF':43, 'LTE_PHY_PUSCH_Power_Control':44, 'LTE_PHY_PUSCH_Tx_Report':45, 'LTE_PHY_RLM_Report':46, 'LTE_PHY_Serv_Cell_Measurement':47, 'LTE_PHY_Serving_Cell_COM_Loop':48, 'LTE_PHY_System_Scan_Results':49, 'LTE_RLC_DL_AM_All_PDU':50, 'LTE_RLC_DL_Config_Log_Packet':51, 'LTE_RLC_DL_Stats':52, 'LTE_RLC_UL_AM_All_PDU':53, 'LTE_RLC_UL_Config_Log_Packet':54, 'LTE_RLC_UL_Stats':55, 'LTE_RRC_CDRX_Events_Info':56, 'LTE_RRC_MIB_Message_Log_Packet':57, 'LTE_RRC_MIB_Packet':58, 'LTE_RRC_OTA_Packet':59, 'LTE_RRC_Serv_Cell_Info':60, 'Srch_TNG_1x_Searcher_Dump':61, 'CDMA_Paging_Channel_Message':62}
support_messages_PHY = {'LTE_PHY_BPLMN_Cell_Confirm':1, 'LTE_PHY_BPLMN_Cell_Request':2, 'LTE_PHY_Connected_Mode_Intra_Freq_Meas':3, 'LTE_PHY_Connected_Mode_Neighbor_Measurement':4, 'LTE_PHY_Idle_Neighbor_Cell_Meas':5, 'LTE_PHY_Inter_RAT_CDMA_Measurement':6, 'LTE_PHY_Inter_RAT_Measurement':7, 'LTE_PHY_PDCCH_Decoding_Result':8, 'LTE_PHY_PDCCH_PHICH_Indication_Report':9, 'LTE_PHY_PDSCH_Decoding_Result':10, 'LTE_PHY_PDSCH_Packet':11, 'LTE_PHY_PDSCH_Stat_Indication':12, 'LTE_PHY_PUCCH_CSF':13, 'LTE_PHY_PUCCH_Power_Control':14, 'LTE_PHY_PUCCH_Tx_Report':15, 'LTE_PHY_PUSCH_CSF':16, 'LTE_PHY_PUSCH_Power_Control':17, 'LTE_PHY_PUSCH_Tx_Report':18, 'LTE_PHY_RLM_Report':19, 'LTE_PHY_Serv_Cell_Measurement':20, 'LTE_PHY_Serving_Cell_COM_Loop':21, 'LTE_PHY_System_Scan_Results':22 }
def convert(llist):
temp = []
for item in llist:
if item != 'Modem_debug_message':
temp.append(support_messages[item])
return temp
def loaddata(filepaths):
allrows = []
for filepath in filepaths:
X = []
data = pd.read_csv(filepath, sep='\t', engine='python')
#select the col 'type_id'
col = data['type_id']
for itm in col:
#if itm != 'Modem_debug_message':
if itm in support_messages_PHY:
X.append(support_messages_PHY[itm])
allrows.append(X)
return allrows
if __name__ == "__main__":
#data = pd.read_csv("./sendsms3/sendsms3.csv", sep='\t', engine='python')
#col = data['type_id']
filepaths = ["./sendsms2/sendsms2.csv", "./sendsms3/sendsms3.csv", "./sendsms4/sendsms4.csv", "./sendsms5/sendsms5.csv", "./receivesms2/receivesms2.csv", "./receivesms3/receivesms3.csv", "./receivesms4/receivesms4.csv", "./receivesms5/receivesms5.csv", "./receivesms6/receivesms6.csv", "./callout1_10s/callout1_10s.csv", "./callout2_10s/callout2_10s.csv", "./callout3_10s/callout3_10s.csv", "./callout4_10s/callout4_10s.csv", "./callout5_10s/callout5_10s.csv", "./receivephonecall_15s/receivephonecall_15s.csv"]
#, "./callin2_10s/callin2_10s.csv", "./callin3_10s/callin3_10s.csv", "./callin4_10s/callin4_10s.csv"]
allrows = loaddata(filepaths)
df = pd.DataFrame(allrows)
#fill n/a by -1
df.fillna(-1, inplace=True)
#convert dataframe to np.array
dfnp = df.to_numpy()
y = np.array([1,1,1,1,1,1,1,1,1,2,2,2,2,2,2])
#clf = svm.SVC(kernel='linear', C=1)
#clf = SVC(C=1, kernel='rbf', gamma=1)
#clf = SVC(gamma='auto')
#clf = RandomForestClassifier(n_estimators=100, max_depth=2,random_state=0)
#clf = tree.DecisionTreeClassifier()
#####################################################
#set lgbm parameters
"""
params = {
'boosting_type': 'gbdt',
'boosting': 'dart',
'objective': 'binary',
#'metric': 'binary_logloss',
'metric': ('auc', 'logloss'),
'learning_rate': 0.01,
'num_leaves':1,
'max_depth':3,
'max_bin':10,
'min_data_in_leaf':8,
'feature_fraction': 0.6,
'bagging_fraction': 1,
'bagging_freq':0,
'lambda_l1': 0,
'lambda_l2': 0,
'min_split_gain': 0
}
"""
evals_result = {}
lgb_train = lgb.Dataset(dfnp, y, free_raw_data=False)
lgb_eval = lgb.Dataset(dfnp, y, reference=lgb_train,free_raw_data=False)
gbm = lgb.train(params,
lgb_train,
#num_boost_round=2000,
num_boost_round=20,
valid_sets=lgb_eval,
evals_result=evals_result,
#early_stopping_rounds=30
) # 早停系数
preds_offline = gbm.predict(dfnp, num_iteration=gbm.best_iteration)
print preds_offline
print evals_result
import matplotlib.pyplot as plt
lgb.create_tree_digraph(gbm, tree_index=1)
"""
ax = lgb.plot_metric(evals_result, metric='auc')#metric的值与之前的params里面的值对应
plt.show()
ax = lgb.plot_tree(gbm, tree_index=0, figsize=(20, 8), show_info=['split_gain'])
plt.show()
"""
########################################################
"""
scores = cross_val_score(clf, dfnp, y, cv=2)
print scores
#test = np.array([[1, 2, 3], [4, 5, 6]])
#y = np.array([1, 2])
clf.fit(dfnp, y)
print clf.score(dfnp, y)
#load test data
test_filepaths = ["./sendsms3/sendsms3.csv", "./sendsms4/sendsms4.csv", "./sendsms5/sendsms5.csv", "./receivesms2/receivesms2.csv", "./receivesms3/receivesms3.csv", "./receivesms4/receivesms4.csv", "./receivesms5/receivesms5.csv", "./receivesms6/receivesms6.csv", "./callout1_10s/callout1_10s.csv", "./callout2_10s/callout2_10s.csv", "./callout3_10s/callout3_10s.csv", "./callout4_10s/callout4_10s.csv", "./callout5_10s/callout5_10s.csv", "./receivephonecall_15s/receivephonecall_15s.csv"]
#, "./callin1_10s/callin1_10s.csv", "./callin2_10s/callin2_10s.csv", "./callin3_10s/callin3_10s.csv", "./callin4_10s/callin4_10s.csv" , "./callin5_10s/callin5_10s.csv"]
df_test = pd.DataFrame(loaddata(test_filepaths))
df_test.fillna(-1, inplace=True)
dfnp_test = df_test.to_numpy()
#print dfnp_test[-1]
print(clf.predict(dfnp_test))
df_test.to_csv('dfnp_test.csv', sep='\t', encoding='utf-8')
"""
"""
with open("iris.dot", 'w') as f:
f = tree.export_graphviz(clf, out_file=f)
"""
[INFO] [OfflineReplayer]: Enable WCDMA_RRC_States
[INFO] [LteNasAnalyzer]: Warning: duplicate analyzer declaration: LteNasAnalyzer
Traceback (most recent call last):
File "dataprepare.py", line 149, in <module>
File "dataprepare.py", line 137, in DataToCSV
File "dataprepare.py", line 54, in ReadRawFile
File "build/bdist.linux-x86_64/egg/mobile_insight/analyzer/lte_nas_analyzer.py", line 59, in __init__
File "build/bdist.linux-x86_64/egg/mobile_insight/analyzer/protocol_analyzer.py", line 44, in __init__
File "build/bdist.linux-x86_64/egg/mobile_insight/analyzer/profile.py", line 191, in __init__
File "build/bdist.linux-x86_64/egg/mobile_insight/analyzer/profile.py", line 238, in __build_db
sqlite3.OperationalError: unable to open database file
I'm trying to decode and NAS messages over LTE using lte-nas-layer-example.py
from the examples
directory but no output on the terminal.
Python code for the same...
#!/usr/bin/python
# Filename: lte-measurement-example
import os
import sys
# Import MobileInsight modules
from mobile_insight.analyzer import *
from mobile_insight.monitor import OnlineMonitor
"""
This example shows how to LTE EMM/ESM layer information with LteNasAnalyzer
"""
if __name__ == "__main__":
if len(sys.argv) < 3:
print "Error: please specify physical port name and baudrate."
print __file__, "SERIAL_PORT_NAME BAUNRATE"
sys.exit(1)
# Initialize a DM monitor
src = OnlineMonitor()
src.set_serial_port(sys.argv[1]) # the serial port to collect the traces
src.set_baudrate(int(sys.argv[2])) # the baudrate of the port
dumper = MsgLogger()
dumper.set_source(src)
dumper.set_decoding(MsgLogger.XML) # decode the message as xml
nas_analyzer = LteNasAnalyzer()
nas_analyzer.set_source(src)
# save the analysis result. All analyzers share the same output file.
dumper.set_log("nas-analyzer-example.txt")
nas_analyzer.set_log("nas-analyzer-example.txt")
# Start the monitoring
src.run()
Shell log from the terminal.
sudo python lte-nas-layer-example.py /dev/ttyUSB0 115200
[INFO] [OnlineMonitor]: Enable collection: LTE_NAS_ESM_OTA_Incoming_Packet
[INFO] [OnlineMonitor]: Enable collection: LTE_NAS_ESM_OTA_Outgoing_Packet
[INFO] [OnlineMonitor]: Enable collection: LTE_NAS_EMM_OTA_Incoming_Packet
[INFO] [OnlineMonitor]: Enable collection: LTE_NAS_EMM_OTA_Outgoing_Packet
[INFO] [OnlineMonitor]: Enable collection: LTE_NAS_EMM_State
[INFO] [OnlineMonitor]: Enable collection: LTE_NAS_ESM_State
PHY COM: /dev/ttyUSB0
PHY BAUD RATE: 115200
What am I doing wrong?
@moonsky219 pls help me solve this issue.
Regards.
Hello, I am aware that the attach reject code is still work in progress, but just notifying that the attach reject value is incorrect.
On line 134 of attach_sr_analyzer.py, it should be field.get("show") == "68"
The value that it is attempting to get is allegedly for 0x41, which is attach request, so the actual value is not correct as well.
I tried online monitor with desktop.
However, some logs seem to be missed or even not recorded.
In my example, I enabled following logs:
src.enable_log('LTE_PHY_Connected_Mode_Intra_Freq_Meas')
src.enable_log('LTE_PHY_PUCCH_Tx_Report')
src.enable_log('LTE_MAC_UL_Tx_Statistics')
src.enable_log('LTE_PHY_PUSCH_Tx_Report')
src.enable_log('LTE_MAC_UL_Buffer_Status_Internal')
I found that 'LTE_PHY_PUSCH_Tx_Report' is missed since the granted TB size from 'LTE_PHY_PUSCH_Tx_Report' is much lower than actual transmitted size and 'LTE_MAC_UL_Buffer_Status_Internal' is not recorded at all.
I attach my online monitoring code and mi2log
mi_report.zip
The issue is related to ws_dissector, it need 2nd level SIB decoding for some messages collected on some devices.
For example, following is the incomplete parsing result of current ws_dissector for one WCDMA signaling message:
<?xml version="1.0" ?>
<dm_log_packet>
<pair key="log_msg_len">47</pair>
<pair key="type_id">WCDMA_RRC_OTA_Packet</pair>
<pair key="timestamp">2017-09-29 19:04:46.017519</pair>
<pair key="Channel Type">4</pair>
<pair key="Radio Bearer ID">40</pair>
<pair key="Message Length">31</pair>
<pair key="Msg" type="list">
<msg>
<packet>
<proto name="frame" pos="0" showname="Frame 0: 39 bytes on wire (312 bits), 39 bytes captured (312 bits)" size="39">
<field name="frame.encap_type" pos="0" show="46" showname="Encapsulation type: USER 1 (46)" size="0"/>
<field name="frame.number" pos="0" show="0" showname="Frame Number: 0" size="0"/>
<field name="frame.len" pos="0" show="39" showname="Frame Length: 39 bytes (312 bits)" size="0"/>
<field name="frame.cap_len" pos="0" show="39" showname="Capture Length: 39 bytes (312 bits)" size="0"/>
<field name="frame.marked" pos="0" show="0" showname="Frame is marked: False" size="0"/>
<field name="frame.ignored" pos="0" show="0" showname="Frame is ignored: False" size="0"/>
<field name="frame.protocols" pos="0" show="user_dlt:aww:rrc" showname="Protocols in frame: user_dlt:aww:rrc" size="0"/>
</proto>
<proto name="user_dlt" pos="0" showname="DLT: 148, Payload: aww (Automator Wireshark Wrapper)" size="39"/>
<proto name="aww" pos="0" showname="Automator Wireshark Wrapper" size="39">
<field name="aww.proto" pos="0" show="104" showname="Protocol: 104" size="4" value="00000068"/>
<field name="aww.data_len" pos="4" show="31" showname="Data length: 31" size="4" value="0000001f"/>
</proto>
<proto name="fake-field-wrapper">
<field name="rrc.BCCH_BCH_Message_element" pos="8" show="" showname="BCCH-BCH-Message" size="30" value="">
<field name="rrc.message_element" pos="8" show="" showname="message" size="30" value="">
<field name="rrc.sfn_Prime" pos="8" show="498" showname="sfn-Prime: 498" size="2" value="3e50"/>
<field hide="yes" name="per.choice_index" pos="9" show="8" showname="Choice Index: 8" size="1" value="50"/>
<field name="rrc.payload" pos="9" show="8" showname="payload: completeAndFirst (8)" size="29" value="500399820d798842fad1a05169130210cafffffa5dc03e254cb1a0a823">
<field name="rrc.completeAndFirst_element" pos="9" show="" showname="completeAndFirst" size="29" value="">
<field hide="yes" name="per.sequence_of_length" pos="9" show="1" showname="Sequence-Of Length: 1" size="1" value="50"/>
<field name="rrc.completeSIB_List" pos="10" show="1" showname="completeSIB-List: 1 item" size="21" value="0399820d798842fad1a05169130210cafffffa5dc0">
<field name="" pos="10" show="Item 0" size="21" value="0399820d798842fad1a05169130210cafffffa5dc0">
<field name="rrc.CompleteSIBshort_element" pos="10" show="" showname="CompleteSIBshort" size="21" value="">
<field hide="yes" name="per.enum_index" pos="10" show="3" showname="Enumerated Index: 3" size="1" value="03"/>
<field name="rrc.sib_Type" pos="10" show="3" showname="sib-Type: systemInformationBlockType3 (3)" size="1" value="03"/>
<field hide="yes" name="per.bit_string_length" pos="11" show="154" showname="Bit String Length: 154" size="1" value="99"/>
<field name="rrc.sib_Data_variable" pos="10" show="82:0d:79:88:42:fa:d1:a0:51:69:13:02:10:ca:ff:ff:fa:5d:c0:00" showname="sib-Data-variable: 820d798842fad1a05169130210cafffffa5dc000 [bit length 154, 6 LSB pad bits]" size="20" value="820d798842fad1a05169130210cafffffa5dc000"/>
</field>
</field>
</field>
<field name="rrc.firstSegment_element" pos="31" show="" showname="firstSegment" size="7" value="">
<field hide="yes" name="per.enum_index" pos="31" show="31" showname="Enumerated Index: 31" size="1" value="3e"/>
<field name="rrc.sib_Type" pos="31" show="31" showname="sib-Type: extensionType (31)" size="1" value="3e"/>
<field name="rrc.seg_Count" pos="31" show="2" showname="seg-Count: 2" size="1" value="3e"/>
<field hide="yes" name="per.bit_string_length" pos="32" show="43" showname="Bit String Length: 43" size="1" value="25"/>
<field name="rrc.sib_Data_variable" pos="31" show="65:8d:05:41:19:40" showname="sib-Data-variable: 658d05411940 [bit length 43, 5 LSB pad bits, 0110 0101 1000 1101 0000 0101 0100 0001 0001 1001 010. .... decimal value 3489261029578]" size="6" value="658d05411940"/>
</field>
</field>
</field>
</field>
</field>
</proto>
</packet>
</msg>
</pair>
</dm_log_packet>
However, the content of systemInformationBlockType3 is not extracted, ws_dissector should somehow continue to parsing the payload: 820d798842fad1a05169130210cafffffa5dc000.
Feeding this binary to ws_dissector gives us more information:
<packet>
<proto name="frame" showname="Frame 0: 28 bytes on wire (224 bits), 28 bytes captured (224 bits)" size="28" pos="0">
<field name="frame.encap_type" showname="Encapsulation type: USER 1 (46)" size="0" pos="0" show="46"/>
<field name="frame.number" showname="Frame Number: 0" size="0" pos="0" show="0"/>
<field name="frame.len" showname="Frame Length: 28 bytes (224 bits)" size="0" pos="0" show="28"/>
<field name="frame.cap_len" showname="Capture Length: 28 bytes (224 bits)" size="0" pos="0" show="28"/>
<field name="frame.marked" showname="Frame is marked: False" size="0" pos="0" show="0"/>
<field name="frame.ignored" showname="Frame is ignored: False" size="0" pos="0" show="0"/>
<field name="frame.protocols" showname="Protocols in frame: user_dlt:aww:rrc" size="0" pos="0" show="user_dlt:aww:rrc"/>
</proto>
<proto name="user_dlt" showname="DLT: 148, Payload: aww (Automator Wireshark Wrapper)" size="28" pos="0"/>
<proto name="aww" showname="Automator Wireshark Wrapper" size="28" pos="0">
<field name="aww.proto" showname="Protocol: 153" size="4" pos="0" show="153" value="00000099"/>
<field name="aww.data_len" showname="Data length: 20" size="4" pos="4" show="20" value="00000014"/>
</proto>
<proto name="fake-field-wrapper">
<field name="rrc.SysInfoType3_element" showname="SysInfoType3" size="19" pos="8" show="" value="">
<field name="per.optional_field_bit" showname="1... .... Optional Field Bit: True (v4b0NonCriticalExtensions is present)" hide="yes" size="1" pos="8" show="1" value="1" unmaskedvalue="82"/>
<field name="rrc.sib4indicator" showname=".0.. .... sib4indicator: False" size="1" pos="8" show="0" value="82"/>
<field name="rrc.cellIdentity" showname="cellIdentity: 0835e620 [bit length 28, 4 LSB pad bits, 0000 1000 0011 0101 1110 0110 0010 .... decimal value 8609378]" size="4" pos="8" show="08:35:e6:20" value="0835e620"/>
<field name="rrc.cellSelectReselectInfo_element" showname="cellSelectReselectInfo" size="11" pos="11" show="" value="">
<field name="per.optional_field_bit" showname=".... ..0. Optional Field Bit: False (mappingInfo is NOT present)" hide="yes" size="1" pos="11" show="0" value="0" unmaskedvalue="88"/>
<field name="per.optional_field_bit" showname=".... ...0 Optional Field Bit: False (hcs-ServingCellInformation is NOT present)" hide="yes" size="1" pos="11" show="0" value="0" unmaskedvalue="88"/>
<field name="per.choice_index" showname="Choice Index: 0" hide="yes" size="1" pos="12" show="0" value="42"/>
<field name="rrc.cellSelectQualityMeasure" showname="cellSelectQualityMeasure: cpich-Ec-N0 (0)" size="1" pos="12" show="0" value="42">
<field name="rrc.cpich_Ec_N0_element" showname="cpich-Ec-N0" size="1" pos="12" show="" value="">
<field name="per.optional_field_bit" showname=".1.. .... Optional Field Bit: True (q-HYST-2-S is present)" hide="yes" size="1" pos="12" show="1" value="1" unmaskedvalue="42"/>
<field name="rrc.q_HYST_2_S" showname="q-HYST-2-S: 1" size="1" pos="12" show="1" value="42"/>
</field>
</field>
<field name="per.choice_index" showname="Choice Index: 0" hide="yes" size="1" pos="12" show="0" value="42"/>
<field name="rrc.modeSpecificInfo" showname="modeSpecificInfo: fdd (0)" size="7" pos="12" show="0" value="42fad1a0516913">
<field name="rrc.fdd_element" showname="fdd" size="6" pos="13" show="" value="">
<field name="per.optional_field_bit" showname="1... .... Optional Field Bit: True (s-Intrasearch is present)" hide="yes" size="1" pos="13" show="1" value="1" unmaskedvalue="fa"/>
<field name="per.optional_field_bit" showname=".1.. .... Optional Field Bit: True (s-Intersearch is present)" hide="yes" size="1" pos="13" show="1" value="1" unmaskedvalue="fa"/>
<field name="per.optional_field_bit" showname="..1. .... Optional Field Bit: True (s-SearchHCS is present)" hide="yes" size="1" pos="13" show="1" value="1" unmaskedvalue="fa"/>
<field name="per.optional_field_bit" showname="...1 .... Optional Field Bit: True (rat-List is present)" hide="yes" size="1" pos="13" show="1" value="1" unmaskedvalue="fa"/>
<field name="rrc.s_Intrasearch" showname="s-Intrasearch: 5" size="1" pos="13" show="5" value="fa"/>
<field name="rrc.s_Intersearch" showname="s-Intersearch: 4" size="1" pos="14" show="4" value="d1"/>
<field name="rrc.s_SearchHCS" showname="s-SearchHCS: -1" size="1" pos="14" show="-1" value="d1"/>
<field name="per.sequence_of_length" showname="Sequence-Of Length: 1" hide="yes" size="1" pos="15" show="1" value="a0"/>
<field name="rrc.rat_List" showname="rat-List: 1 item" size="2" pos="16" show="1" value="5169">
<field name="" show="Item 0" size="2" pos="16" value="5169">
<field name="rrc.RAT_FDD_Info_element" showname="RAT-FDD-Info" size="2" pos="16" show="" value="">
<field name="per.optional_field_bit" showname=".1.. .... Optional Field Bit: True (s-HCS-RAT is present)" hide="yes" size="1" pos="16" show="1" value="1" unmaskedvalue="51"/>
<field name="per.enum_index" showname="Enumerated Index: 0" hide="yes" size="1" pos="16" show="0" value="51"/>
<field name="rrc.rat_Identifier" showname="rat-Identifier: gsm (0)" size="1" pos="16" show="0" value="51"/>
<field name="rrc.s_SearchRAT" showname="s-SearchRAT: 1" size="1" pos="16" show="1" value="51"/>
<field name="rrc.s_HCS_RAT" showname="s-HCS-RAT: -1" size="1" pos="17" show="-1" value="69"/>
<field name="rrc.s_Limit_SearchRAT" showname="s-Limit-SearchRAT: 1" size="1" pos="17" show="1" value="69"/>
</field>
</field>
</field>
<field name="rrc.q_QualMin" showname="q-QualMin: -18" size="1" pos="18" show="-18" value="13"/>
<field name="rrc.q_RxlevMin" showname="q-RxlevMin: -57" size="1" pos="19" show="-57" value="02"/>
</field>
</field>
<field name="rrc.q_Hyst_l_S" showname="q-Hyst-l-S: 1" size="1" pos="19" show="1" value="02"/>
<field name="rrc.t_Reselection_S" showname="t-Reselection-S: 1" size="1" pos="20" show="1" value="10"/>
<field name="rrc.maxAllowedUL_TX_Power" showname="maxAllowedUL-TX-Power: 24" size="1" pos="21" show="24" value="ca"/>
</field>
<field name="rrc.cellAccessRestriction_element" showname="cellAccessRestriction" size="2" pos="22" show="" value="">
<field name="per.optional_field_bit" showname="1... .... Optional Field Bit: True (accessClassBarredList is present)" hide="yes" size="1" pos="22" show="1" value="1" unmaskedvalue="ff"/>
<field name="per.choice_index" showname="Choice Index: 1" hide="yes" size="1" pos="22" show="1" value="ff"/>
<field name="rrc.cellBarred" showname="cellBarred: notBarred (1)" size="1" pos="22" show="1" value="ff">
<field name="rrc.notBarred_element" showname="notBarred: NULL" size="1" pos="22" show="" value=""/>
</field>
<field name="per.enum_index" showname="Enumerated Index: 1" hide="yes" size="1" pos="22" show="1" value="ff"/>
<field name="rrc.cellReservedForOperatorUse" showname="cellReservedForOperatorUse: notReserved (1)" size="1" pos="22" show="1" value="ff"/>
<field name="per.enum_index" showname="Enumerated Index: 1" hide="yes" size="1" pos="22" show="1" value="ff"/>
<field name="rrc.cellReservationExtension" showname="cellReservationExtension: notReserved (1)" size="1" pos="22" show="1" value="ff"/>
<field name="rrc.accessClassBarredList" showname="accessClassBarredList: 16 items" size="2" pos="22" show="16" value="ffff">
<field name="" show="Item 0" size="1" pos="22" value="ff">
<field name="per.enum_index" showname="Enumerated Index: 1" hide="yes" size="1" pos="22" show="1" value="ff"/>
<field name="rrc.AccessClassBarred" showname="AccessClassBarred: notBarred (1)" size="1" pos="22" show="1" value="ff"/>
</field>
<field name="" show="Item 1" size="1" pos="22" value="ff">
<field name="per.enum_index" showname="Enumerated Index: 1" hide="yes" size="1" pos="22" show="1" value="ff"/>
<field name="rrc.AccessClassBarred" showname="AccessClassBarred: notBarred (1)" size="1" pos="22" show="1" value="ff"/>
</field>
<field name="" show="Item 2" size="1" pos="22" value="ff">
<field name="per.enum_index" showname="Enumerated Index: 1" hide="yes" size="1" pos="22" show="1" value="ff"/>
<field name="rrc.AccessClassBarred" showname="AccessClassBarred: notBarred (1)" size="1" pos="22" show="1" value="ff"/>
</field>
<field name="" show="Item 3" size="1" pos="22" value="ff">
<field name="per.enum_index" showname="Enumerated Index: 1" hide="yes" size="1" pos="22" show="1" value="ff"/>
<field name="rrc.AccessClassBarred" showname="AccessClassBarred: notBarred (1)" size="1" pos="22" show="1" value="ff"/>
</field>
<field name="" show="Item 4" size="1" pos="23" value="ff">
<field name="per.enum_index" showname="Enumerated Index: 1" hide="yes" size="1" pos="23" show="1" value="ff"/>
<field name="rrc.AccessClassBarred" showname="AccessClassBarred: notBarred (1)" size="1" pos="23" show="1" value="ff"/>
</field>
<field name="" show="Item 5" size="1" pos="23" value="ff">
<field name="per.enum_index" showname="Enumerated Index: 1" hide="yes" size="1" pos="23" show="1" value="ff"/>
<field name="rrc.AccessClassBarred" showname="AccessClassBarred: notBarred (1)" size="1" pos="23" show="1" value="ff"/>
</field>
<field name="" show="Item 6" size="1" pos="23" value="ff">
<field name="per.enum_index" showname="Enumerated Index: 1" hide="yes" size="1" pos="23" show="1" value="ff"/>
<field name="rrc.AccessClassBarred" showname="AccessClassBarred: notBarred (1)" size="1" pos="23" show="1" value="ff"/>
</field>
<field name="" show="Item 7" size="1" pos="23" value="ff">
<field name="per.enum_index" showname="Enumerated Index: 1" hide="yes" size="1" pos="23" show="1" value="ff"/>
<field name="rrc.AccessClassBarred" showname="AccessClassBarred: notBarred (1)" size="1" pos="23" show="1" value="ff"/>
</field>
<field name="" show="Item 8" size="1" pos="23" value="ff">
<field name="per.enum_index" showname="Enumerated Index: 1" hide="yes" size="1" pos="23" show="1" value="ff"/>
<field name="rrc.AccessClassBarred" showname="AccessClassBarred: notBarred (1)" size="1" pos="23" show="1" value="ff"/>
</field>
<field name="" show="Item 9" size="1" pos="23" value="ff">
<field name="per.enum_index" showname="Enumerated Index: 1" hide="yes" size="1" pos="23" show="1" value="ff"/>
<field name="rrc.AccessClassBarred" showname="AccessClassBarred: notBarred (1)" size="1" pos="23" show="1" value="ff"/>
</field>
<field name="" show="Item 10" size="1" pos="23" value="ff">
<field name="per.enum_index" showname="Enumerated Index: 1" hide="yes" size="1" pos="23" show="1" value="ff"/>
<field name="rrc.AccessClassBarred" showname="AccessClassBarred: notBarred (1)" size="1" pos="23" show="1" value="ff"/>
</field>
<field name="" show="Item 11" size="1" pos="23" value="ff">
<field name="per.enum_index" showname="Enumerated Index: 1" hide="yes" size="1" pos="23" show="1" value="ff"/>
<field name="rrc.AccessClassBarred" showname="AccessClassBarred: notBarred (1)" size="1" pos="23" show="1" value="ff"/>
</field>
<field name="" show="Item 12" size="1" pos="24" value="fa">
<field name="per.enum_index" showname="Enumerated Index: 1" hide="yes" size="1" pos="24" show="1" value="fa"/>
<field name="rrc.AccessClassBarred" showname="AccessClassBarred: notBarred (1)" size="1" pos="24" show="1" value="fa"/>
</field>
<field name="" show="Item 13" size="1" pos="24" value="fa">
<field name="per.enum_index" showname="Enumerated Index: 1" hide="yes" size="1" pos="24" show="1" value="fa"/>
<field name="rrc.AccessClassBarred" showname="AccessClassBarred: notBarred (1)" size="1" pos="24" show="1" value="fa"/>
</field>
<field name="" show="Item 14" size="1" pos="24" value="fa">
<field name="per.enum_index" showname="Enumerated Index: 1" hide="yes" size="1" pos="24" show="1" value="fa"/>
<field name="rrc.AccessClassBarred" showname="AccessClassBarred: notBarred (1)" size="1" pos="24" show="1" value="fa"/>
</field>
<field name="" show="Item 15" size="1" pos="24" value="fa">
<field name="per.enum_index" showname="Enumerated Index: 1" hide="yes" size="1" pos="24" show="1" value="fa"/>
<field name="rrc.AccessClassBarred" showname="AccessClassBarred: notBarred (1)" size="1" pos="24" show="1" value="fa"/>
</field>
</field>
</field>
<field name="rrc.v4b0NonCriticalExtensions_element" showname="v4b0NonCriticalExtensions" size="3" pos="24" show="" value="">
<field name="per.optional_field_bit" showname=".... 1... Optional Field Bit: True (v590NonCriticalExtension is present)" hide="yes" size="1" pos="24" show="1" value="1" unmaskedvalue="fa"/>
<field name="rrc.sysInfoType3_v4b0ext_element" showname="sysInfoType3-v4b0ext" size="1" pos="24" show="" value="">
<field name="per.optional_field_bit" showname=".... .0.. Optional Field Bit: False (mapping-LCR is NOT present)" hide="yes" size="1" pos="24" show="0" value="0" unmaskedvalue="fa"/>
</field>
<field name="rrc.v590NonCriticalExtension_element" showname="v590NonCriticalExtension" size="3" pos="24" show="" value="">
<field name="per.optional_field_bit" showname=".... ..1. Optional Field Bit: True (v5c0NoncriticalExtension is present)" hide="yes" size="1" pos="24" show="1" value="1" unmaskedvalue="fa"/>
<field name="rrc.sysInfoType3_v590ext_element" showname="sysInfoType3-v590ext" size="1" pos="24" show="" value="">
<field name="per.optional_field_bit" showname=".... ...0 Optional Field Bit: False (cellSelectReselectInfo-v590ext is NOT present)" hide="yes" size="1" pos="24" show="0" value="0" unmaskedvalue="fa"/>
</field>
<field name="rrc.v5c0NoncriticalExtension_element" showname="v5c0NoncriticalExtension" size="2" pos="25" show="" value="">
<field name="per.optional_field_bit" showname="0... .... Optional Field Bit: False (v670NonCriticalExtension is NOT present)" hide="yes" size="1" pos="25" show="0" value="0" unmaskedvalue="5d"/>
<field name="rrc.sysInfoType3_v5c0ext_element" showname="sysInfoType3-v5c0ext" size="2" pos="25" show="" value="">
<field name="per.optional_field_bit" showname=".1.. .... Optional Field Bit: True (cellSelectReselectInfoTreselectionScaling-v5c0ext is present)" hide="yes" size="1" pos="25" show="1" value="1" unmaskedvalue="5d"/>
<field name="rrc.cellSelectReselectInfoTreselectionScaling_v5c0ext_element" showname="cellSelectReselectInfoTreselectionScaling-v5c0ext" size="2" pos="25" show="" value="">
<field name="per.optional_field_bit" showname="..0. .... Optional Field Bit: False (non-HCS-t-CR-Max is NOT present)" hide="yes" size="1" pos="25" show="0" value="0" unmaskedvalue="5d"/>
<field name="per.optional_field_bit" showname="...1 .... Optional Field Bit: True (speedDependentScalingFactor is present)" hide="yes" size="1" pos="25" show="1" value="1" unmaskedvalue="5d"/>
<field name="per.optional_field_bit" showname=".... 1... Optional Field Bit: True (interFrequencyTreselectionScalingFactor is present)" hide="yes" size="1" pos="25" show="1" value="1" unmaskedvalue="5d"/>
<field name="per.optional_field_bit" showname=".... .1.. Optional Field Bit: True (interRATTreselectionScalingFactor is present)" hide="yes" size="1" pos="25" show="1" value="1" unmaskedvalue="5d"/>
<field name="rrc.speedDependentScalingFactor" showname="speedDependentScalingFactor: 7" size="1" pos="25" show="7" value="5d"/>
<field name="rrc.interFrequencyTreselectionScalingFactor" showname="interFrequencyTreselectionScalingFactor: 4" size="1" pos="26" show="4" value="c0"/>
<field name="rrc.interRATTreselectionScalingFactor" showname="interRATTreselectionScalingFactor: 4" size="1" pos="26" show="4" value="c0"/>
</field>
</field>
</field>
</field>
</field>
</field>
</proto>
</packet>
I am importing an MsgLogger
in interactive Python shell.
⇒ python
Python 2.7.10 (default, Jul 15 2017, 17:16:57)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.getcwd()
'/Users/shihaoxu/dev/MobileInsight-3.2.0'
>>> from mobile_insight.analyzer import MsgLogger
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "mobile_insight/__init__.py", line 9, in <module>
import analyzer
File "mobile_insight/analyzer/__init__.py", line 51, in <module>
from log_analyzer import LogAnalyzer
File "mobile_insight/analyzer/log_analyzer.py", line 15, in <module>
from mobile_insight.monitor import OfflineReplayer
File "mobile_insight/monitor/__init__.py", line 26, in <module>
from dm_collector import * # P4A: THIS LINE WILL BE DELETED ###
File "mobile_insight/monitor/dm_collector/__init__.py", line 10, in <module>
from dm_collector import DMCollector # P4A: THIS LINE WILL BE DELETED ###
File "mobile_insight/monitor/dm_collector/dm_collector.py", line 22, in <module>
import dm_collector_c
ImportError: No module named dm_collector_c
It tried to import a dm_collector_c
module and failed to find it.
Also, I can't understand why we can import dm_collector_c
.
Because from my view, the directory dm_collector_c
has only .h
.cpp
files.
No __init.py
file exitsts.
Does this indicate I didn't install mobile-insight-core successfully?
I encountered the following logs while parsing three message types (version).
(MI)Unknown LTE PHY PDSCH Demapper Configuration version: 0x16
(MI)Unknown LTE PHY PUSCH Tx Report version: 0x16
(MI)Unknown LTE PHY PUCCH CSF version: 0x15
We are trying to use Mobileinsight in order to get cell reselection info for 2g/3g/4g for Google Pixel 3a. We've managed to receive this info for 2g and 4g. But for some reason we can't receive it for 3g.
To be more exact, we are receiving WCDMA_Signaling_Messages. But inside of that message we never receive RRC_COMPLETE_SIB byte. And therefore we don't receive RRC_SIB3 part with all the reselection info that we need.
Can you please advice what else can we try to receive those messages? :)
when execute all of python file in the examples, it will present the same prob"OSError: [Errno 2] No such file or directory"
python offline-analysis-filtering.py
python monitor-example.py 8888 2000
and so on
Traceback (most recent call last):
File "lte-measurement-example.py", line 23, in <module>
src = OnlineMonitor()
File "build/bdist.macosx-10.14-x86_64/egg/mobile_insight/monitor/online_monitor.py", line 156, in __init__
File "build/bdist.macosx-10.14-x86_64/egg/mobile_insight/monitor/dm_collector/dm_collector.py", line 49, in __init__
File "build/bdist.macosx-10.14-x86_64/egg/mobile_insight/monitor/dm_collector/dm_endec/dm_log_packet.py", line 439, in init
File "build/bdist.macosx-10.14-x86_64/egg/mobile_insight/monitor/dm_collector/dm_endec/ws_dissector.py", line 104, in init_proc
File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 394, in __init__
errread, errwrite)
File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1047, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
We were trying to set up mobileinsight-core
using install-ubuntu.sh
on a fresh Debian 10 install (and reproduced the issue on a fresh Xubuntu 20.04). Running cmake
on line 47 failed. When tracking down the cause, we noticed three problems:
cmake
command and is a bit misleading, one could think that the cmake
binary is being looked for on the wrong path. Could be changed to: "Error when executing 'cmake -DBUILD_wireshark=OFF .' in ${WIRESHARK_SRC_PATH}/"
cmake
is suppressed (i.e., redirected to /dev/null
, including stderr
) which makes troubleshooting harder than it has to be. Why is this done actually, the other commands log to the standard outputs as well?cmake
, build-essential
, git
, and libgcrypt20-dev
(could be added to line 35)While the last point is the main problem, it might be worth to consider fixing the others as well.
nick@nick-VirtualBox:~/mobileinsight-core-master$ ./install-ubuntu.sh
** Installer Script for mobileinsight-core on Ubuntu **
Author : Zengwen Yuan (zyuan [at] cs.ucla.edu), Haotian Deng (deng164 [at] purdue.edu)
Date : 2020-10-20
Rev : 4.0
Usage : ./install-ubuntu.sh
[…]
Installing dependencies for compiling Wireshark libraries
[sudo] password for nick:
Reading package lists... Done
Building dependency tree
Reading state information... Done
pkg-config is already the newest version (0.29.1-0ubuntu4).
pkg-config set to manually installed.
wget is already the newest version (1.20.3-1ubuntu1).
wget set to manually installed.
The following additional packages will be installed:
binutils binutils-common binutils-x86-64-linux-gnu cpp-9 gcc gcc-10-base gcc-9 gcc-9-base libasan5
libatomic1 libbinutils libblkid-dev libblkid1 libc-dev-bin libc6 libc6-dbg libc6-dev libcc1-0 libcrypt-dev
libctf-nobfd0 libctf0 libffi-dev libfl-dev libfl2 libgcc-9-dev libgcc-s1 libgfortran5 libglib2.0-0
libglib2.0-bin libglib2.0-dev-bin libgomp1 libitm1 liblsan0 libmount-dev libmount1 libpcap0.8-dev
libpcre16-3 libpcre2-16-0 libpcre2-dev libpcre2-posix2 libpcre3-dev libpcre32-3 libpcrecpp0v5 libquadmath0
libselinux1-dev libsepol1-dev libsigsegv2 libstdc++6 libtsan0 libubsan1 libuuid1 linux-libc-dev m4
manpages-dev python3-distutils python3-lib2to3 uuid-dev zlib1g zlib1g-dev
[…]
Checking Wireshark sources to compile ws_dissector
You do not have source codes for Wireshark version 3.2.7, downloading...
--2021-06-18 15:01:49-- https://www.wireshark.org/download/src/all-versions/wireshark-3.2.7.tar.xz
Resolving www.wireshark.org (www.wireshark.org)... 104.26.11.240, 104.26.10.240, 172.67.75.39, ...
Connecting to www.wireshark.org (www.wireshark.org)|104.26.11.240|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 31659996 (30M) [application/x-xz]
Saving to: ‘wireshark-3.2.7.tar.xz’
wireshark-3.2.7.tar.xz 100%[==========================================>] 30,19M 13,8MB/s in 2,2s
2021-06-18 15:01:52 (13,8 MB/s) - ‘wireshark-3.2.7.tar.xz’ saved [31659996/31659996]
Configuring Wireshark sources for ws_dissector compilation...
Error when executing '/home/nick/mobileinsight-core-master/wireshark-3.2.7/cmake -DBUILD_wireshark .'
You need to manually fix it before continuation. Exiting with status 3
After installing all the required dependencies when I run "vagrant up" this is the error I get:
mammar:~/Desktop/idp/mobileinsight-dev-1.2/MI_FOLDER$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Box 'bento/ubuntu-16.04' could not be found. Attempting to find and install...
default: Box Provider: virtualbox
default: Box Version: 201708.22.0
==> default: Loading metadata for box 'bento/ubuntu-16.04'
default: URL: https://vagrantcloud.com/bento/ubuntu-16.04
The box you're attempting to add has no available version that
matches the constraints you requested. Please double-check your
settings. Also verify that if you specified version constraints,
that the provider you wish to use is available for these constraints.
Box: bento/ubuntu-16.04
Address: https://vagrantcloud.com/bento/ubuntu-16.04
Constraints: 201708.22.0
Available versions: 201801.02.0, 201802.02.0, 201803.24.0, 201806.08.0, 201807.12.0, 201808.24.0, 201812.15.0, 201812.27.0, 201906.18.0, 201910.20.0, 201910.21.0, 201910.31.0, 201912.03.0, 201912.04.0, 201912.15.0, 202002.04.0
Could you please tell me what the problem would be and how I could solve it.
On Google Pixel 2 XL phone with T-mobile, US (310260), MobileInsight cannot support newest version of the LTE RRC OTA packet and LTE PHY PDSCH Demapper Configuration version
11-22 12:59:10.180 13167 13181 I python2.7: 2017-11-22,12:59:10.180132 [b][color=00ff00][INFO][/color] [OnlineMonitor][/b]: Enable collection: LTE_PHY_Inter_RAT_Measurement
11-22 12:59:11.350 13167 13181 I python : (MI)Unknown LTE RRC OTA packet version: 15
11-22 12:59:12.545 13167 13181 I python : (MI)Unknown LTE PHY PDSCH Demapper Configuration version: 0x7a
There is a TypeError exception when the MsgLogger uses the dictionary (MsgLogger.DICT) as decoding format.
The log file used is: MobileInsight-6.0.0b0/examples/offline_log_example.mi2log
#!/usr/bin/python
import os
import sys
from mobile_insight.monitor import OfflineReplayer
from mobile_insight.analyzer import MsgLogger, LteRrcAnalyzer, LteNasAnalyzer, LtePhyAnalyzer, LteMacAnalyzer, LtePdcpAnalyzer, LteRlcAnalyzer
if __name__ == "__main__":
if len(sys.argv) != 2:
print("USE: python3 test.py <FILE.mi2log>")
sys.exit()
# Initialize a 3G/4G monitor
src = OfflineReplayer()
src.set_input_path(sys.argv[1])
logger = MsgLogger()
logger.set_decode_format(MsgLogger.DICT)
logger.set_dump_type(MsgLogger.STDIO_ONLY)
logger.set_source(src)
# Analyzers
#lte_rrc_analyzer = LteRrcAnalyzer()
#lte_rrc_analyzer.set_source(src) # bind with the monitor
#lte_nas_analyzer = LteNasAnalyzer()
#lte_nas_analyzer.set_source(src)
lte_phy_analyzer = LtePhyAnalyzer()
lte_phy_analyzer.set_source(src)
#lte_mac_analyzer = LteMacAnalyzer()
#lte_mac_analyzer.set_source(src)
# Start the monitoring
src.run()
[INFO] [OfflineReplayer]: Enable LTE_PHY_PDSCH_Packet
[INFO] [OfflineReplayer]: Enable LTE_PHY_PUSCH_CSF
[INFO] [OfflineReplayer]: Enable LTE_MAC_UL_Tx_Statistics
[INFO] [OfflineReplayer]: Enable LTE_PHY_PUCCH_Tx_Report
[INFO] [OfflineReplayer]: Enable LTE_PHY_PUSCH_Tx_Report
[INFO] [OfflineReplayer]: STARTED: 1619524127.593187
[INFO] [OfflineReplayer]: Loading MobileInsight-6.0.0b0/examples/offline_log_example.mi2log
[INFO] [OfflineReplayer]: Loading: 1619524127.596473
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/MobileInsight-6.0.0b0-py3.6-linux-x86_64.egg/mobile_insight/monitor/offline_replayer.py", line 209, in run
self.send(event)
File "/usr/local/lib/python3.6/dist-packages/MobileInsight-6.0.0b0-py3.6-linux-x86_64.egg/mobile_insight/element.py", line 104, in send
self.to_list[i].recv(self, event)
File "/usr/local/lib/python3.6/dist-packages/MobileInsight-6.0.0b0-py3.6-linux-x86_64.egg/mobile_insight/analyzer/analyzer.py", line 234, in recv
self.source_callback[i](event)
File "/usr/local/lib/python3.6/dist-packages/MobileInsight-6.0.0b0-py3.6-linux-x86_64.egg/mobile_insight/analyzer/msg_logger.py", line 134, in __dump_message
self.log_info(decoded_msg)
File "/usr/local/lib/python3.6/dist-packages/MobileInsight-6.0.0b0-py3.6-linux-x86_64.egg/mobile_insight/element.py", line 146, in log_info
+ self.__class__.__name__ + ']\033[0m: ' + msg
TypeError: must be str, not dict
Versions
Description
Please add support for LTE_RRC_OTA_Packet version: 26 I am facing the following error.
Logs
(MI)Unknown LTE RRC OTA packet version: 26
attached is the mi2log file.
When execute gui file by ·./mi-guiafter successfully installing MobileInsight-3.4.0 core, it shows
ImportError: No module named mobile_insight.analyzer`
cd MobileInsight-3.4.0
sh install-macos.sh # successfully installed while with lots of warnings
cd gui
./mi-gui
Errors shows after executing ./mi-gui
:
Traceback (most recent call last):
File "/usr/local/share/mobileinsight/mobile_insight_gui.py", line 25, in <module>
from mobile_insight.analyzer import LogAnalyzer
ImportError: No module named mobile_insight.analyzer
This error is similar to the warning shows in the installation process, which is [WARNING] Failed to run offline analysis example!
.
Do I really install the MobileInsight correctly?
Although there exist many warnings, it shows that Congratulations! mobileinsight-core is successfully installed!
. I'm so confused.
In file included from dm_collector_c/export_manager.cpp:14:
dm_collector_c/log_packet_helper.h:191:39: error: call to 'pow' is ambiguous
temp = (int)buffer[0] & ((int)std::pow(2, left_remaining) - 1);
^~~~~~~~
/usr/include/c++/4.2.1/cmath:365:3: note: candidate function
pow(double __x, int __i)
^
/usr/include/c++/4.2.1/cmath:369:3: note: candidate function
pow(float __x, int __n)
^
/usr/include/c++/4.2.1/cmath:373:3: note: candidate function
pow(long double __x, int __n)
^
/usr/include/math.h:443:15: note: candidate function
extern double pow(double, double);
^
/usr/include/c++/4.2.1/cmath:357:3: note: candidate function
pow(float __x, float __y)
^
/usr/include/c++/4.2.1/cmath:361:3: note: candidate function
pow(long double __x, long double __y)
^
In file included from dm_collector_c/export_manager.cpp:14:
dm_collector_c/log_packet_helper.h:192:22: error: call to 'pow' is ambiguous
rt += temp * std::pow(2, bitlength - left_remaining);
^~~~~~~~
/usr/include/c++/4.2.1/cmath:365:3: note: candidate function
pow(double __x, int __i)
^
/usr/include/c++/4.2.1/cmath:369:3: note: candidate function
pow(float __x, int __n)
^
/usr/include/c++/4.2.1/cmath:373:3: note: candidate function
pow(long double __x, int __n)
^
/usr/include/math.h:443:15: note: candidate function
extern double pow(double, double);
^
/usr/include/c++/4.2.1/cmath:357:3: note: candidate function
pow(float __x, float __y)
^
/usr/include/c++/4.2.1/cmath:361:3: note: candidate function
pow(long double __x, long double __y)
^
In file included from dm_collector_c/export_manager.cpp:14:
dm_collector_c/log_packet_helper.h:201:29: error: call to 'pow' is ambiguous
temp = temp & ((int)std::pow(2, bitlength) - 1);
^~~~~~~~
/usr/include/c++/4.2.1/cmath:365:3: note: candidate function
pow(double __x, int __i)
^
/usr/include/c++/4.2.1/cmath:369:3: note: candidate function
pow(float __x, int __n)
^
/usr/include/c++/4.2.1/cmath:373:3: note: candidate function
pow(long double __x, int __n)
^
/usr/include/math.h:443:15: note: candidate function
extern double pow(double, double);
^
/usr/include/c++/4.2.1/cmath:357:3: note: candidate function
pow(float __x, float __y)
^
/usr/include/c++/4.2.1/cmath:361:3: note: candidate function
pow(long double __x, long double __y)
^
3 errors generated.
error: command '/usr/bin/clang' failed with exit status 1
Traceback (most recent call last):
File "offline-analysis-example.py", line 11, in <module>
from mobile_insight.monitor import OfflineReplayer
ImportError: No module named mobile_insight.monitor
I get the following error when installing on ubuntu 22.04:
** Installer Script for mobileinsight-core on Ubuntu **
Author : Zengwen Yuan (zyuan [at] cs.ucla.edu), Haotian Deng (deng164 [at] purdue.edu), Yuanjie Li (yuanjiel [at] tsinghua.edu.cn)
Date : 2020-10-20
Rev : 4.0
Usage : ./install-ubuntu.sh
Upgrading MobileInsight...
./uninstall.sh: line 12: python: command not found
Uninstalling MobileInsight-core v2.0 from your computer
This script will require sudo access to delete following files:
/home/mohsen/Programs/mobileinsight-core-master/MobileInsight.egg-info
/home/mohsen/Programs/mobileinsight-core-master/build
/home/mohsen/Programs/mobileinsight-core-master/dist
/home/mohsen/Programs/mobileinsight-core-master/ws_dissector/ws_dissector
/home/mohsen/Programs/mobileinsight-core-master/wireshark-2.0.*
/home/mohsen/.cache/Python-Eggs/MobileInsight*
/home/mohsen/.python-eggs/MobileInsight*
/usr/local/bin/mi-gui
/usr/local/bin/ws_dissector
/usr/local/share/mobileinsight/*
/usr/lib/libwireshark.so*
/usr/lib/libwsutil.so*
/usr/lib/libwireshark.6.dylib
/usr/lib/libwireshark.dylib
/usr/lib/libwiretap.5.dylib
/usr/lib/libwiretap.dylib
/usr/lib/libwsutil.6.dylib
/usr/lib/libwsutil.dylib
Uninstalling old MobileInsight build files and libs...
Uninstallation finished.
Installing dependencies for compiling Wireshark libraries
[sudo] password for mohsen:
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Note, selecting 'libgcrypt20-dev' instead of 'libgcrypt-dev'
Package qt5-default is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
E: Package 'qt5-default' has no installation candidate
Checking Wireshark sources to compile ws_dissector
Configuring Wireshark sources for ws_dissector compilation...
Error when executing '/home/mohsen/Programs/mobileinsight-core-master/wireshark-3.4.0/cmake --disable-wireshark .'
You need to manually fix it before continuation. Exiting with status 3
When trying to run plugin DEMO_PHY_ONLINE or DEMO_PHY_OFFLINE (from a log recorded with NetLoger) with the French MNO "Free" I get errors because the plugin (or the core?) doesn't handle some message types or some protocol versions.
Importing the log to the desktop app. clairly show the problem.
#!/usr/bin/python
# Filename: offline-analysis-filtering.py
import os
import sys
"""
Offline analysis: save the log as a new one with pre-defined filter
"""
#Import MobileInsight modules
from mobile_insight.monitor import OfflineReplayer
if __name__ == "__main__":
# Initialize a 3G/4G monitor
src = OfflineReplayer()
src.set_input_path("./offline_log_example.mi2log")
# Configure the log to be saved
src.enable_log("LTE_PHY_PDSCH_Packet")
# Save log as
src.save_log_as("./filtered_log.mi2log")
#Start the monitoring
src.run()
$ python offline-analysis-filtering.py
{}
[INFO] [OfflineReplayer]: Enable LTE_PHY_PDSCH_Packet
[INFO] [OfflineReplayer]: Loading ./offline_log_example.mi2log
Unknown LTE LL1 PDSCH Demapper Configuration version: 0x1
...
-----------------------
$ python offline-analysis-example.py
{}
[INFO] [OfflineReplayer]: Enable LTE_RRC_OTA_Packet
[INFO] [OfflineReplayer]: Enable LTE_RRC_Serv_Cell_Info
[WARNING] [OfflineReplayer]: Unsupported log message type: LTE_RRC_CDRX_Events_Info
[INFO] [OfflineReplayer]: Enable LTE_RRC_CDRX_Events_Info
Traceback (most recent call last):
File "offline-analysis-example.py", line 28, in <module>
lte_rrc_analyzer.set_source(src) #bind with the monitor
File "/usr/local/lib/python2.7/dist-packages/mobile_insight/analyzer/lte_rrc_analyzer.py", line 847, in set_source
source.enable_log("LTE_RRC_CDRX_Events_Info")
File "/usr/local/lib/python2.7/dist-packages/mobile_insight/monitor/offline_replayer.py", line 126, in enable_log
dm_collector_c.set_filtered(self._type_names)
ValueError: Wrong type name.
------------------------
openning the same log file with Mobileinsight-gui ;
Unknown LTE LL1 PDSCH Demapper Configuration version: 0x1
Unknown LTE LL1 PDSCH Demapper Configuration version: 0x1
Unknown LTE LL1 PDSCH Demapper Configuration version: 0x1
Unknown LTE LL1 PDSCH Demapper Configuration version: 0x1
Unknown LTE LL1 PDSCH Demapper Configuration version: 0x1
Unknown LTE LL1 PDSCH Demapper Configuration version: 0x1
Unknown LTE LL1 PDSCH Demapper Configuration version: 0x1
Unknown LTE LL1 PDSCH Demapper Configuration version: 0x1
Unknown LTE LL1 PDSCH Demapper Configuration version: 0x1
Unknown LTE ML1 CMLIFMR version: 0x2
Unknown LTE ML1 CMLIFMR version: 0x2
Unknown LTE ML1 CMLIFMR version: 0x2
Unknown LTE ML1 CMLIFMR version: 0x2
Unknown LTE ML1 CMLIFMR version: 0x2
Unknown LTE ML1 CMLIFMR version: 0x2
Unknown LTE ML1 CMLIFMR version: 0x2
Unknown LTE ML1 CMLIFMR version: 0x2
Unknown LTE ML1 CMLIFMR version: 0x2
Unknown LTE LL1 PDSCH Demapper Configuration version: 0x1
Unknown LTE ML1 CMLIFMR version: 0x2
Unknown LTE ML1 CMLIFMR version: 0x2
Unknown LTE ML1 CMLIFMR version: 0x2
...
Testing the MobileInsight offline analysis example. Traceback (most recent call last): File "offline-analysis-example.py", line 11, in <module> from mobile_insight.monitor import OfflineReplayer ModuleNotFoundError: No module named 'mobile_insight' Failed to run offline analysis example! Exiting with status 4.
Immediate cause:
git clone
includes the Mach bundle dm_collector_c.so
, which causes the offline-analysis test in install-ubuntu.sh
to fail.
`file ./mobileinsight-core/mobile_insight/monitor/dm_collector/dm_collector_c.so
./mobileinsight-core/mobile_insight/monitor/dm_collector/dm_collector_c.so: Mach-O 64-bit x86_64 bundle, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|WEAK_DEFINES|BINDS_TO_WEAK>`
There is no target to build dm_collector_c.so
under Ubuntu.
Hello, is it possible to add Wireshark live view or some way to export the saved logs to Wireshark?
Thanks.
Hi, nice project ,but some trouble when i use MobileInsight GUI:
// REPLACE ME: What are you trying to get done, what has happened, what went wrong, and what did you expect?
mobileinsight-core-3.0/gui ./mi-gui
Traceback (most recent call last):
File "/usr/local/share/mobileinsight/mobile_insight_gui.py", line 12, in <module>
import wx.animate
ImportError: No module named animate
can you tell me how to fix?thanks!
Please add support for LTE_PHY_Serv_Cell_Measurement version 18. I am facing the following error
(MI)Unknown LTE PHY Subpacket version: 0x19 - 18
and following is the json it creates, with empty Subpackets array.
{
"log_msg_len": 112,
"timestamp": "2019-03-08 18:08:00.021941",
"type_id": "LTE_PHY_Serv_Cell_Measurement",
"Subpackets": [],
"Version": 1,
"Number of SubPackets": 1
}
Please add support for LTE_PHY_Serv_Cell_Measurement version 24. I am facing the following error
Logs
(MI)Unknown LTE PHY Subpacket version: 0x19 - 24
json created
LTE_PHY_Serv_Cell_Measurement
timestamp:2019-03-29 14:39:01.921850
Version 1
Number of SubPackets: 1
Subpackets: list
diag_log_20190329_103908_5662db1e88c26169678919616f6a59e3_motorola-motog(6)_310260.zip
From the latest version, 6.0.0-beta I mean, I find the only handover log info is in lte_rrc_analyzer.py and there is nothing related in nr_rrc_analyzer.py, so I want to make sure that does beta version support NR handoff message display(NSA exactly)? Like 5G to 5G, 5G to 4G, 4G to 5G?
When the latest version of mobileinsight-core is installed and run the python API - offline analysis url: > http://www.mobileinsight.net/tutorial-api.html#offline-analysis
receiving an error as indicated below:
Traceback (most recent call last):
File "test.py", line 4, in
from mobile_insight.monitor import OfflineReplayer
File "/home/ubuntu/mobileinsight-core/mobile_insight/init.py", line 9, in
import analyzer
File "/home/ubuntu/mobileinsight-core/mobile_insight/analyzer/init.py", line 48, in
from log_analyzer import LogAnalyzer
File "/home/ubuntu/mobileinsight-core/mobile_insight/analyzer/log_analyzer.py", line 15, in
from mobile_insight.monitor import OfflineReplayer
File "/home/ubuntu/mobileinsight-core/mobile_insight/monitor/init.py", line 26, in
from dm_collector import * # P4A: THIS LINE WILL BE DELETED ###
File "/home/ubuntu/mobileinsight-core/mobile_insight/monitor/dm_collector/init.py", line 10, in
from dm_collector import DMCollector # P4A: THIS LINE WILL BE DELETED ###
File "/home/ubuntu/mobileinsight-core/mobile_insight/monitor/dm_collector/dm_collector.py", line 22, in
import dm_collector_c
ImportError: No module named dm_collector_c
import os
import sys
from mobile_insight.monitor import OfflineReplayer
from mobile_insight.analyzer import LteRrcAnalyzer,WcdmaRrcAnalyzer
src = OfflineReplayer()
# Load offline logs
src.set_input_path("./offline_log_example.mi2log")
# RRC analyzer
lte_rrc_analyzer = LteRrcAnalyzer()
lte_rrc_analyzer.set_source(src) #bind with the monitor
wcdma_rrc_analyzer = WcdmaRrcAnalyzer()
wcdma_rrc_analyzer.set_source(src) #bind with the monitor
src.run()
When I visualize the logs collected from my android, it shows a modem debug message every 10 seconds. Is that a common phenomenon?
All the other messages time is correct, only the modem debug message's time is 1980-01-06.
<?xml version=« 1.0 » ?>
<dm_log_packet>
<pair key=« type_id »>Modem_debug_message</pair>
<pair key=« timestamp »>1980-01-06 00:00:00</pair>
<pair key=« Version »>146</pair>
<pair key=« Number of parameters »>4</pair>
<pair key=« Timestamp »>1980-01-06 00:00:00</pair>
<pair key=« Line of code »>4674</pair>
</dm_log_packet>
Hi ,
The modem check is done through getprop - where qualacomm is identified through MSM or MDM.
With Qualacomm snapdragon 845 , value is
[ro.board.platform]: [sdm845]
Therefore app is unable to work properly.
Running offline-analysis-example.py results in an error.
This is also seen with other log files.
python3 offline-analysis-example.py
Init RRC Analyzer
[INFO] [OfflineReplayer]: Enable LTE_RRC_OTA_Packet
[INFO] [OfflineReplayer]: Enable LTE_RRC_Serv_Cell_Info
[INFO] [OfflineReplayer]: Enable LTE_RRC_CDRX_Events_Info
[INFO] [WcdmaRrcAnalyzer]: Initialing WcdmaRrcAnalyzer..
[INFO] [OfflineReplayer]: Enable WCDMA_RRC_OTA_Packet
[INFO] [OfflineReplayer]: Enable WCDMA_RRC_Serv_Cell_Info
[INFO] [OfflineReplayer]: Enable WCDMA_RRC_States
[INFO] [OfflineReplayer]: Enable LTE_NAS_ESM_OTA_Incoming_Packet
[INFO] [OfflineReplayer]: Enable LTE_NAS_ESM_OTA_Outgoing_Packet
[INFO] [OfflineReplayer]: Enable LTE_NAS_EMM_OTA_Incoming_Packet
[INFO] [OfflineReplayer]: Enable LTE_NAS_EMM_OTA_Outgoing_Packet
[INFO] [OfflineReplayer]: Enable LTE_NAS_EMM_State
[INFO] [OfflineReplayer]: Enable LTE_NAS_ESM_State
[INFO] [UmtsNasAnalyzer]: Initialing UmtsNasAnalyzer..
[INFO] [OfflineReplayer]: Enable UMTS_NAS_OTA_Packet
[INFO] [OfflineReplayer]: Enable UMTS_NAS_GMM_State
[INFO] [OfflineReplayer]: Enable UMTS_NAS_MM_State
[INFO] [OfflineReplayer]: Enable UMTS_NAS_MM_REG_State
[INFO] [OfflineReplayer]: Enable LTE_PHY_PDSCH_Packet
[INFO] [OfflineReplayer]: Enable LTE_PHY_PUSCH_CSF
[INFO] [OfflineReplayer]: Enable LTE_MAC_UL_Tx_Statistics
[INFO] [OfflineReplayer]: Enable LTE_PHY_PUCCH_Tx_Report
[INFO] [OfflineReplayer]: Enable LTE_PHY_PUSCH_Tx_Report
[INFO] [OfflineReplayer]: Enable LTE_MAC_UL_Buffer_Status_Internal
[INFO] [OfflineReplayer]: Enable LTE_PHY_PDSCH_Stat_Indication
[INFO] [OfflineReplayer]: STARTED: 1619740498.4493096
[INFO] [OfflineReplayer]: Loading ./offline_log_example.mi2log
[INFO] [OfflineReplayer]: Loading: 1619740498.449642
[INFO] [UmtsNasAnalyzer]: Find One UMTS_NAS_MM_State
[INFO] [UmtsNasAnalyzer]: MmStatus MM.state=MM_IDLE MM.substate=MM_NO_CELL_AVAILABLE MM.update_status=UPDATED PLMN=unknown LAC=unknown RAC=unknown Network_operation_mode=unknown CS/PS_service_type=unknown
[INFO] [UmtsNasAnalyzer]: MM State: MM_IDLE
[INFO] [UmtsNasAnalyzer]: Find One UMTS_NAS_GMM_State
[INFO] [UmtsNasAnalyzer]: GMM State: GMM_DEREGISTERED
[INFO] [LteNasAnalyzer]: EmmStatus EMM.state=EMM_DEREGISTERED EMM.substate=EMM_DEREGISTERED_ATTACH_NEEDED MCC=310 MNC=410 MMEGI=0x50ff MMEC=0x16 TMSI=0x77fd95c0
[INFO] [LteNasAnalyzer]: Call flow status: IDLE
[INFO] [LteNasAnalyzer]: EMM state: EMM_DEREGISTERED
[INFO] [UmtsNasAnalyzer]: Find One UMTS_NAS_GMM_State
[INFO] [UmtsNasAnalyzer]: Find One UMTS_NAS_GMM_State
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/MobileInsight-5.0.0-py3.8-linux-x86_64.egg/mobile_insight/monitor/offline_replayer.py", line 209, in run
self.send(event)
File "/usr/local/lib/python3.8/dist-packages/MobileInsight-5.0.0-py3.8-linux-x86_64.egg/mobile_insight/element.py", line 104, in send
self.to_list[i].recv(self, event)
File "/usr/local/lib/python3.8/dist-packages/MobileInsight-5.0.0-py3.8-linux-x86_64.egg/mobile_insight/analyzer/analyzer.py", line 234, in recv
self.source_callback[i](event)
File "/usr/local/lib/python3.8/dist-packages/MobileInsight-5.0.0-py3.8-linux-x86_64.egg/mobile_insight/analyzer/lte_rrc_analyzer.py", line 249, in __rrc_filter
self.__callback_sib_config(xml_msg)
File "/usr/local/lib/python3.8/dist-packages/MobileInsight-5.0.0-py3.8-linux-x86_64.egg/mobile_insight/analyzer/lte_rrc_analyzer.py", line 407, in __callback_sib_config
int(field_val['lte-rrc.utra_q_RxLevMin']) * 2,
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'
I'm installing the MobileInsight3.4 by get into directory and execute sh install-macos.sh
.
Then it shows some warnings.
cd MobileInsight-3.4.0
sh install-macos.sh
Warnings are:
[WARNING] MobileInsight relies on correctly configured Python environment by Homebrew (not the Python that comes with macOS).
It appears that Homebrew reported some issues in your environment. Please carefully review and fix them if necessary before proceeding.
If the Python environment is not correctly configured, MobileInsight may not work properly.
If you feel that the warning reported by Homebrew is not related, you may ignore it.
Are you sure that you want to ignore the warnings reported by Homebrew and continue? [y/N] MobileInsight installation canceled due to Homebrew reported warning. Please fix those issues and run MobileInsight installation again.
Exiting with status 2.
OS: Ubuntu 18.04
Python version: Python 2.7
Python installation method: apt
I have followed the instructions mentioned in STEP 3 here to every detail : http://www.mobileinsight.net/get_started_desktop.html. The serial of the phone shows up as ttyUSB0. The online monitor does not capture any packets despite the user activity performed on the phone even though the apk managed to detect it. We even ran the monitor-example.py file and got no packet. Can someone please help me figure this out
// REPLACE ME: Paste your code here.
mobileinsight-core-3.0/example/monitor-example.py
python monitor-example.py /dev/ttyUSB0 9600
[INFO] [OnlineMonitor]: Enable collection: LTE_RRC_OTA_Packet
PHY COM: /dev/ttyUSB0
PHY BAUD RATE: 9600
Disable logs
Enable logs
KeyboardInterrupt Detected: Disabling all logs
I let the smartphone connect to the desktop, and I used monitor-example.py to monitor the messages from the smartphone.
However, some packets have errors and cannot be decoded.
desktop: Ubuntu 20.04
smartphone: Samsung A42
Error:
(MI)Unknown LTE RRC OTA packet version: 27
Please add support for LTE_PHY_Connected_Mode_Neighbour_Measurement version 24. I am facing the following error
Logs
(MI)Unknown LTE PHY Connected Neighbor Cell Meas subpkt id and version: 31 - 24
mi-gui output
?xml version="1.0" ?>
<dm_log_packet>
"log_msg_len" 180
LTE_PHY_Connected_Mode_Neighbor_Measurement
"timestamp" 2019-03-29 14:39:02.255147
"Version" 1
"Number of SubPackets" 2
"SubPackets" type = "list"
"SubPacket ID" 30
"Version" 3
"SubPacket Size" 48
"SubPacket ID" 31
"Version" 24
"SubPacket Size" 116
diag_log_20190329_103908_5662db1e88c26169678919616f6a59e3_motorola-motog(6)_310260.zip
Experiment Setting
Description
I got NR MAC UL TB Stats of major version 1, minor version 1 & NR MAC UL Physical Channel Schedule Report of major version 1 and minor version 4 which is not implemented in the source code.
Because of this error, I can't parse packet information correctly.
Could you please add support for these packets?
Or please let me know where can I find the information on the packet's format versions.
Thanks.
Could someone please help me how to add new messages (i.e. 2g messages)? I want to capture all 2g messages
When connecting to Quectel EC25 module through DM port.
How could one find out what is the format of this new version, so that I try to update the code?
Another question would be that some other commands, like for example LTE_MAC_UL_Transport_Block don't have any outputs. Does it mean that the modem software is implementing some limitation, or they are not at all available in the modem , or ?
Hello,
I collected a fairly big (few GB) mi2log file over several hours and would like to use the OfflineReplayer to decode this into a file (json format). Unfortunately, the RAM fills up fairly quickly (>16GB) and then my PC crashes.
Thus, I suspect there might be a memory leak in the code, since it seems like the memory used for decoding previous messages is not freed up again.
As a workaround for me it might also be good for me if I could split the mi2log file into several smaller files. Is this possible?
Thanks and kind regards!
I tried to use offline-latency-analysis-ul.py to do uplink latency analysis, but I don't know the meaning of stats.cum_err_block. in the code. Can anyone help what this is and how it impacts to uplink latency?
FC
Versions
Description
Please add support for LTE_PHY_Serv_Cell_Measurement version 48. I am facing the following error
Logs
(MI)Unknown LTE PHY Subpacket version: 0x19 - 48
Attached is the log file
testfile.qmdl.gz
Thanks.
Hi,
I use MI v3.4 to collect logs from Android phone (OnePlus 5T with Android 7.1) and use latest desktop mobileinsight-core to analyze the data. However, when I add lte_phy_analyzer, I receive this error message
src = OfflineReplayer()
src.set_input_path(log_filename)
lte_phy_analyzer = LtePhyAnalyzer()
lte_phy_analyzer.set_source(src)
src.run()
(MI)Unknown LTE PHY PUCCH Tx Report version: 121
Traceback (most recent call last):
File "build/bdist.linux-x86_64/egg/mobile_insight/monitor/offline_replayer.py", line 205, in run
self.send(event)
File "build/bdist.linux-x86_64/egg/mobile_insight/element.py", line 108, in send
map(G, self.to_list)
File "build/bdist.linux-x86_64/egg/mobile_insight/element.py", line 107, in G
def G(module): return module.recv(self, event)
File "build/bdist.linux-x86_64/egg/mobile_insight/analyzer/analyzer.py", line 231, in recv
map(G, self.source_callback)
File "build/bdist.linux-x86_64/egg/mobile_insight/analyzer/analyzer.py", line 227, in G
def G(f): return f(event)
File "build/bdist.linux-x86_64/egg/mobile_insight/analyzer/lte_phy_analyzer.py", line 364, in __msg_callback
self.callback_pucch(msg)
File "build/bdist.linux-x86_64/egg/mobile_insight/analyzer/lte_phy_analyzer.py", line 120, in callback_pucch
records = log_item['Records']
KeyError: 'Records'
I got the following errors when I run ./install-ubuntu.sh.
Same error when I run python3 ./offline-analysis-example.py
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/MobileInsight-5.0.0-py3.8-linux-x86_64.egg/mobile_insight/monitor/offline_replayer.py", line 209, in run
self.send(event)
File "/usr/local/lib/python3.8/dist-packages/MobileInsight-5.0.0-py3.8-linux-x86_64.egg/mobile_insight/element.py", line 104, in send
self.to_list[i].recv(self, event)
File "/usr/local/lib/python3.8/dist-packages/MobileInsight-5.0.0-py3.8-linux-x86_64.egg/mobile_insight/analyzer/analyzer.py", line 234, in recv
self.source_callbacki
File "/usr/local/lib/python3.8/dist-packages/MobileInsight-5.0.0-py3.8-linux-x86_64.egg/mobile_insight/analyzer/lte_rrc_analyzer.py", line 249, in __rrc_filter
self.__callback_sib_config(xml_msg)
File "/usr/local/lib/python3.8/dist-packages/MobileInsight-5.0.0-py3.8-linux-x86_64.egg/mobile_insight/analyzer/lte_rrc_analyzer.py", line 451, in __callback_sib_config
int(field_val['lte-rrc.utra_q_RxLevMin']) * 2,
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'
Failed to run offline analysis example!
Exiting with status 4.
Any idea how to fix it?
Thanks,
execute ./install-ubuntu.sh
While installing:
Installed /usr/local/lib/python2.7/dist-packages/MobileInsight-4.0.0-py2.7-linux-x86_64.egg
Processing dependencies for MobileInsight==4.0.0
Finished processing dependencies for MobileInsight==4.0.0
Installing GUI for MobileInsight...
Testing the MobileInsight offline analysis example.
Traceback (most recent call last):
File "offline-analysis-example.py", line 11, in
from mobile_insight.monitor import OfflineReplayer
ImportError: No module named mobile_insight.monitor
Failed to run offline analysis example!
Exiting with status 4.
When using in my own pythonfile:
import mobile_insight
Traceback (most recent call last):
File "", line 1, in
File "mobile_insight/init.py", line 9, in
import analyzer
File "mobile_insight/analyzer/init.py", line 48, in
from log_analyzer import LogAnalyzer
File "mobile_insight/analyzer/log_analyzer.py", line 15, in
from mobile_insight.monitor import OfflineReplayer
File "mobile_insight/monitor/init.py", line 26, in
from dm_collector import * # P4A: THIS LINE WILL BE DELETED ###
File "mobile_insight/monitor/dm_collector/init.py", line 10, in
from dm_collector import DMCollector # P4A: THIS LINE WILL BE DELETED ###
File "mobile_insight/monitor/dm_collector/dm_collector.py", line 22, in
import dm_collector_c
ImportError: No module named dm_collector_c
Homebrew recently updated their way of handling Python on macOS:
https://docs.brew.sh/Homebrew-and-Python.html
Essentially they do not install at /usr/local/bin/python
for Python 2 anymore. They use python2
and pip2
for the installed Python.
macOS installation script needs updates to adapt to this change.
I have a Quectel EC25 modem connected to a linux PC. and am able to use monitor-example.py in mobileinsight to capture logs with this.
However, python3 ./offline-analysis-example.py monitor-example.mi2log to parse it, I got the following output and it seems not right. The log is the attachment for purpose of reproducing this issue.
[INFO] [OfflineReplayer]: Enable LTE_RRC_OTA_Packet
[INFO] [OfflineReplayer]: Enable LTE_RRC_Serv_Cell_Info
[INFO] [OfflineReplayer]: Enable LTE_RRC_CDRX_Events_Info
[INFO] [WcdmaRrcAnalyzer]: Initialing WcdmaRrcAnalyzer..
[INFO] [OfflineReplayer]: Enable WCDMA_RRC_OTA_Packet
[INFO] [OfflineReplayer]: Enable WCDMA_RRC_Serv_Cell_Info
[INFO] [OfflineReplayer]: Enable WCDMA_RRC_States
[INFO] [OfflineReplayer]: Enable LTE_NAS_ESM_OTA_Incoming_Packet
[INFO] [OfflineReplayer]: Enable LTE_NAS_ESM_OTA_Outgoing_Packet
[INFO] [OfflineReplayer]: Enable LTE_NAS_EMM_OTA_Incoming_Packet
[INFO] [OfflineReplayer]: Enable LTE_NAS_EMM_OTA_Outgoing_Packet
[INFO] [OfflineReplayer]: Enable LTE_NAS_EMM_State
[INFO] [OfflineReplayer]: Enable LTE_NAS_ESM_State
[INFO] [UmtsNasAnalyzer]: Initialing UmtsNasAnalyzer..
[INFO] [OfflineReplayer]: Enable UMTS_NAS_OTA_Packet
[INFO] [OfflineReplayer]: Enable UMTS_NAS_GMM_State
[INFO] [OfflineReplayer]: Enable UMTS_NAS_MM_State
[INFO] [OfflineReplayer]: Enable UMTS_NAS_MM_REG_State
[INFO] [OfflineReplayer]: Enable LTE_PHY_PDSCH_Packet
[INFO] [OfflineReplayer]: Enable LTE_PHY_PUSCH_CSF
[INFO] [OfflineReplayer]: Enable LTE_MAC_UL_Tx_Statistics
[INFO] [OfflineReplayer]: Enable LTE_PHY_PUCCH_Tx_Report
[INFO] [OfflineReplayer]: Enable LTE_PHY_PUSCH_Tx_Report
[INFO] [OfflineReplayer]: Enable LTE_MAC_UL_Buffer_Status_Internal
[INFO] [OfflineReplayer]: Enable LTE_PHY_PDSCH_Stat_Indication
[INFO] [OfflineReplayer]: STARTED: 1632789290.0979013
[INFO] [OfflineReplayer]: Loading ./offline_log_example.mi2log
[INFO] [OfflineReplayer]: Loading: 1632789290.097962
[INFO] [UmtsNasAnalyzer]: Find One UMTS_NAS_MM_State
[INFO] [UmtsNasAnalyzer]: MmStatus MM.state=MM_IDLE MM.substate=MM_NO_CELL_AVAILABLE MM.update_status=UPDATED PLMN=unknown LAC=unknown R
AC=unknown Network_operation_mode=unknown CS/PS_service_type=unknown
[INFO] [UmtsNasAnalyzer]: MM State: MM_IDLE
[INFO] [UmtsNasAnalyzer]: Find One UMTS_NAS_GMM_State
[INFO] [UmtsNasAnalyzer]: GMM State: GMM_DEREGISTERED
[INFO] [LteNasAnalyzer]: EmmStatus EMM.state=EMM_DEREGISTERED EMM.substate=EMM_DEREGISTERED_ATTACH_NEEDED MCC=310 MNC=410 MMEGI=0x50ff M
MEC=0x16 TMSI=0x77fd95c0
[INFO] [LteNasAnalyzer]: Call flow status: IDLE
[INFO] [LteNasAnalyzer]: EMM state: EMM_DEREGISTERED
[INFO] [UmtsNasAnalyzer]: Find One UMTS_NAS_GMM_State
[INFO] [UmtsNasAnalyzer]: Find One UMTS_NAS_GMM_State
[INFO] [LteRrcAnalyzer]: LteRrcStatus cellID=405 frequency=5780 TAC=34624 connected=False
[INFO] [LteNasAnalyzer]: EmmStatus EMM.state=EMM_DEREGISTERED EMM.substate=EMM sub-state = 7 MCC=310 MNC=410 MMEGI=0x50ff MMEC=0x16 TMSI
=0x77fd95c0
[INFO] [UmtsNasAnalyzer]: Find One UMTS_NAS_GMM_State
[INFO] [UmtsNasAnalyzer]: Find One UMTS_NAS_GMM_State
[INFO] [LteNasAnalyzer]: EmmStatus EMM.state=EMM_REGISTERED_INITIATED EMM.substate=EMM_WAITING_FOR_NW_RESPONSE MCC=310 MNC=410 MMEGI=0x5
0ff MMEC=0x16 TMSI=0x77fd95c0
[INFO] [LteNasAnalyzer]: EMM state: EMM_REGISTERED_INITIATED
[INFO] [LteNasAnalyzer]: CSFB Capbility: True
[INFO] [LteNasAnalyzer]: EsmQos peak_tput=unknown mean_tput=unknown max_bitrate_ulink=unknown max_bitrate_dlink=unknown guaranteed_birat
e_ulink=unknown guaranteed_birate_dlink=unknown max_bitrate_ulink_ext=unknown max_bitrate_dlink_ext=unknown guaranteed_birate_ulink_ext=unknown guaranteed_
birate_dlink_ext=unknown
[INFO] [LteNasAnalyzer]: EsmQos delivery_order=unknown traffic_class=unknown QCI=unknown delay_class=unknown transfer_delay=unknown resi
dual_BER=unknown
[INFO] [LtePhyAnalyzer]: PUSCH_TX_POWER: {'tx power': 22, 'timestamp': '2016-03-23 21:55:49.302584'}
[INFO] [LtePhyAnalyzer]: PUCCH_TX_POWER: {'tx power': 1, 'timestamp': '2016-03-23 21:55:49.322533'}
[INFO] [LtePhyAnalyzer]: PUSCH_TX_POWER: {'tx power': 13, 'timestamp': '2016-03-23 21:55:49.342538'}
[INFO] [LtePhyAnalyzer]: PUCCH_TX_POWER: {'tx power': 1, 'timestamp': '2016-03-23 21:55:49.342538'}
[INFO] [LtePhyAnalyzer]: SR_EVENT: {'timestamp': '2016-03-23 21:55:49.342538', 'fn and subfn': 5738}
[INFO] [LtePhyAnalyzer]: PUSCH_CQI: {'WideBand CQI CW0': '10', 'WideBand CQI CW1': '0'}
[INFO] [LtePhyAnalyzer]: PUSCH_TX_POWER: {'tx power': 16, 'timestamp': '2016-03-23 21:55:49.362534'}
[INFO] [LtePhyAnalyzer]: PUSCH_TX_POWER: {'tx power': 17, 'timestamp': '2016-03-23 21:55:49.362534'}
[INFO] [LtePhyAnalyzer]: PUCCH_TX_POWER: {'tx power': 1, 'timestamp': '2016-03-23 21:55:49.362534'}
[INFO] [LtePhyAnalyzer]: PUCCH_TX_POWER: {'tx power': 1, 'timestamp': '2016-03-23 21:55:49.362534'}
Init RRC Analyzer
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/MobileInsight-5.0.0-py3.8-linux-x86_64.egg/mobile_insight/monitor/offline_replayer.py", line 209, in run
self.send(event)
File "/usr/local/lib/python3.8/dist-packages/MobileInsight-5.0.0-py3.8-linux-x86_64.egg/mobile_insight/element.py", line 104, in send
self.to_list[i].recv(self, event)
File "/usr/local/lib/python3.8/dist-packages/MobileInsight-5.0.0-py3.8-linux-x86_64.egg/mobile_insight/analyzer/analyzer.py", line 234, in recv
self.source_callbacki
File "/usr/local/lib/python3.8/dist-packages/MobileInsight-5.0.0-py3.8-linux-x86_64.egg/mobile_insight/analyzer/lte_rrc_analyzer.py", line 249, in _rrc
filter
self.__callback_sib_config(xml_msg)
File "/usr/local/lib/python3.8/dist-packages/MobileInsight-5.0.0-py3.8-linux-x86_64.egg/mobile_insight/analyzer/lte_rrc_analyzer.py", line 451, in __call
back_sib_config
int(field_val['lte-rrc.utra_q_RxLevMin']) * 2,
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'
Thanks,
I want to use "generate_diag_cfg" call to save my enabled logs to a Diag.cfg file.
Is there an example or an instruction of how to do this?
Thanks,
When a LTE_RRC_OTA_Message which contains long content of UL_DCCH/UECapabilityInformation arrives, the ws_dissector.py get blocked by its for loop which trying to read output from ws_dissector. ws_dissector failed to finish the decoding even after 30 minutes.
The UECapabilityInformation messages has 2092 bytes in payload.
// REPLACE ME: What are you trying to get done, what has happened, what went wrong, and what did you expect?
// REPLACE ME: Paste your code here.
// REPLACE ME: Paste the command line logs here.
Right now I have cross-compiled a newest set of the Wireshark 2.4.1 and Glib 2.54.0 libraries for Android.
To update to these libraries, the Wireshark APIs used by the ws_dissector
also needs to be updated accordingly. (Remember that the API used in Wireshark 2.0.x stable release and 2.4.x stable release changed a bit)
when running the install-macos.sh
script,
brew gives the warning that
Warning: You have unlinked kegs in your Cellar
Leaving kegs unlinked can lead to build-trouble and cause brews that depend on
those kegs to fail to run properly once built. Run `brew link` on these:
lz4
but if I brew link lz4
, it shows
⇒ brew link lz4
Linking /usr/local/Cellar/lz4/1.8.0...
Error: Could not symlink bin/lz4
Target /usr/local/bin/lz4
is a symlink belonging to [email protected]. You can unlink it:
brew unlink [email protected]
To force the link and overwrite all conflicting files:
brew link --overwrite lz4
To list all files that would be deleted:
brew link --overwrite --dry-run lz4
lz4 is already linked to [email protected]
When I tried to convert an offline mi2log file to xml, I met with an Import Error, which said no module named request. However urllib is a python3 package.
Should I modify the code and still use python2? Because when I tried to use python3, I met with another traceback.
import os
import sys
from mobile_insight.monitor import OfflineReplayer
from mobile_insight.analyzer import LteRrcAnalyzer, WcdmaRrcAnalyzer
from mobile_insight.analyzer import MsgLogger
src = OfflineReplayer()
path = input("Please input the path of mi2log you want to convert: ")
src.set_input_path(path)
dumper = MsgLogger()
dumper.set_source(src)
dumper.set_decoding(MsgLogger.XML)
src.run()
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.