[BUG] Issue with messaging, image files & audio's don't send or receive at all. about session-ios HOT 13 OPEN

No worries, I did end up adding a couple of tweaks and some more logs around this in #946 (version 2.4.5) so once that is released (hopefully next week) if you do run into this issue or #938 again then if you can export and send through logs shortly after seeing the issue(s) it will hopefully help us narrow down what is happening

from session-ios.

@featherridge Can you please share your device logs? I'm not able to re-produce this issue on my side

from session-ios.

@mpretty-cyro sent through to help desk. Subject attention: mpretty-cyro.

let me know if you need anything else.

from session-ios.

@mpretty-cyro sent through to help desk. Subject attention: mpretty-cyro.

let me know if you need anything else.

@featherridge thanks for that - did you experience the issue during the 20 minute period these logs were showing?

Taking a look at the logs I can see a couple of 502 errors related to files, but there are also 404 errors for the same file ids and can't see any logs for MessageSendJob/AttachmentUploadJob/AttachmentDownloadJob failures which makes me think the 502 errors are a red herring and these particular files are actually expired display pictures

Aside from that I can see some logs around multiple MessageSendJob and NotifyPushServer jobs being scheduled which look a little questionable but I'll need to add in some more logs in 2.4.5 to debug the issue further

When you do run into the issue do your messages just have the Sending status indefinitely or do they change to Failed to send?

from session-ios.

@featherridge thanks for that - did you experience the issue during the 20 minute period these logs were showing?

cannot recall, i'm sorry.

When you do run into the issue do your messages just have the Sending status indefinitely or do they change to Failed to send?

Either "sending" status perpetually, followed by "failed to send", generally.

not in relation to your questions but -
in regards to the receiving issue, audios perpetually load, whilst never actually loading. this is common.

from session-ios.

@mpretty-cyro
Regarding audios perpetually loading, please see 2 x log when this recently occurred. Attached below.

Increasing number of users experiencing this issue of audios perpetually loading. (Audios continue to spin, never actually being able to play)

Thank you.

com.loki-project.loki-messenger 2024-02-03--05-15-02-938.log
com.loki-project.loki-messenger_2024-02-03--01-25-09-780.log

from session-ios.

@mpretty-cyro Regarding audios perpetually loading, please see 2 x log when this recently occurred. Attached below.

Increasing number of users experiencing this issue of audios perpetually loading. (Audios continue to spin, never actually being able to play)

Thank you.

com.loki-project.loki-messenger 2024-02-03--05-15-02-938.log com.loki-project.loki-messenger_2024-02-03--01-25-09-780.log

Hey @featherridge, thanks for these logs, taking a look at the a few things are jumping out:

I've gone through the logs and can see a couple of things happening:

• There are a few 404 errors in both logs
  • This is generally seen after attachments have expired or a user replaced (or stopped refreshing) their display picture as both have the same 14 day lifetime as messages (we have a task with designs sitting in our backlog to indicate when an attachment failed to download due to this situation)
  • For actual message attachments you generally wouldn't see this issue if it initially failed to upload the attachment as the server returns the fileId as part of the upload response
• There are also a few 502 errors in the second set of logs
  • These are split between "Bad gateway" and "next node not found" errors
  • This indicates a bad service node, in these cases we flag the service node and if it fails 3 times we remove it from the pool your device uses to make requests (there are logs indicating this happened so it should be recovering)

Looking a bit deeper, the first set of logs don't seem to download or upload any attachments, the second set had 6 attachments to download and seems to have decrypted all six

• There is an odd 404 file error around the time of the first attachment but then it successfully decrypted so I assume that was related to a display picture instead (I've reworked the display pic downloading in #894 which should give better logs for these cases when that eventually gets released, but should be unrelated to this issue)
• One of the attachments failed on the first try (with a 502 as mentioned in the above section) but then retried and succeeded ~10 seconds later

Does this match what you saw in the app? (I'm wondering if the issue could be related to the client not correctly handling the downloaded/decrypted files correctly rather than the downloading)

from session-ios.

@mpretty-cyro Regarding audios perpetually loading, please see 2 x log when this recently occurred. Attached below.
Increasing number of users experiencing this issue of audios perpetually loading. (Audios continue to spin, never actually being able to play)
Thank you.
com.loki-project.loki-messenger 2024-02-03--05-15-02-938.log com.loki-project.loki-messenger_2024-02-03--01-25-09-780.log

Hey @featherridge, thanks for these logs, taking a look at the a few things are jumping out:

I've gone through the logs and can see a couple of things happening:

• There are a few 404 errors in both logs

  • This is generally seen after attachments have expired or a user replaced (or stopped refreshing) their display picture as both have the same 14 day lifetime as messages (we have a task with designs sitting in our backlog to indicate when an attachment failed to download due to this situation)
  • For actual message attachments you generally wouldn't see this issue if it initially failed to upload the attachment as the server returns the fileId as part of the upload response

• There are also a few 502 errors in the second set of logs

  • These are split between "Bad gateway" and "next node not found" errors
  • This indicates a bad service node, in these cases we flag the service node and if it fails 3 times we remove it from the pool your device uses to make requests (there are logs indicating this happened so it should be recovering)

Looking a bit deeper, the first set of logs don't seem to download or upload any attachments, the second set had 6 attachments to download and seems to have decrypted all six

• There is an odd 404 file error around the time of the first attachment but then it successfully decrypted so I assume that was related to a display picture instead (I've reworked the display pic downloading in [WIP] Groups Rebuild #894 which should give better logs for these cases when that eventually gets released, but should be unrelated to this issue)
• One of the attachments failed on the first try (with a 502 as mentioned in the above section) but then retried and succeeded ~10 seconds later

Does this match what you saw in the app? (I'm wondering if the issue could be related to the client not correctly handling the downloaded/decrypted files correctly rather than the downloading)

Hi @mpretty-cyro

Does this match what you saw in the app? (I'm wondering if the issue could be related to the client not correctly handling the downloaded/decrypted files correctly rather than the downloading)

My apologies but, these logs are not mine.
What I do know is that these logs were received after the two users both experienced issues with audios perpetually loading (and never actually loading)

Please let me know if you need anything else.

from session-ios.

@featherridge sorry I haven't been able to reproduce this issue on my side so far and nothing really jumps out as an issue in the current logs you've shared.

Based on the logs so far the only guess I have is that for some reason the UI isn't updating once the download completes, but I'd need some more info from the users that experienced the issue to debug and track down why (it could also be due to network issues, or unsupported attachment formats being sent).

The next time you or one of these other users runs into the issue can you please ask them to:
• Provide the logs (again)
• Provide a brief description of the behaviour they saw (ie. Did it ever finish downloading? How many times did it go into the "failure" state? Do they know how much time passed between when the message was sent and when their device tried to download it?)
• If the download has been perpetually loading for ~1 minute, can they restart the app and check whether it continues to perpetually load?

from session-ios.

Hi @mpretty-cyro please see answers and an additional log attached below.

The next time you or one of these other users runs into the issue can you please ask them to:

• Provide the logs (again)
Please see a new attached log.

• Provide a brief description of the behaviour they saw (ie. Did it ever finish downloading? How many times did it go into the "failure" state? Do they know how much time passed between when the message was sent and when their device tried to download it?)
I received a voice message in a chat.
It perpetually loaded (I believe it is not the UI, as even if i click to play it, it does not play, and just continues to spin, same issue does occur on windows, perpetually loads/graphic spins, and I cannot play/download the audio)

• If the download has been perpetually loading for ~1 minute, can they restart the app and check whether it continues to perpetually load?
The voice message perpetually loaded for a few minutes, closed app, restarted, audio file still not playable.
The log was extracted 5-10 mins after the audio was attempted to be played.
It has now been 12~ hours since receiving the audio log, have closed and opened the app multiple times, and it is still not loading/playable.

Thank you.

com.loki-project.loki-messenger 2024-02-19--22-36-53-251.log

from session-ios.

I think the same or similar issue is reproducible on Android by opening a chat with a voice message or picture or video in, and then closing Session before the voice message or picture or video has fully loaded. When you try opening the chat again the voice message or picture or video will never load! In short voice messages, pictures and videos never load if they don't load the first time. I always have problems loading pictures or videos or voice messages when a WiFi connection is unstable.

from session-ios.

Thanks for providing more info @featherridge

I've looked through the logs and have a pretty good understanding of what the issues are:

• In the logs the voice message download is actually failing to download (due to a 502 error - it automatically retries to download 3 times, but all fail)
• Currently the Voice Message UI doesn't have an "error state" so it will appear to load indefinitely if it fails to download
  • I believe each platform will have this issue
  • We have updated designs for attachment downloading which would resolve this but they are prioritised below a few larger issues at the moment
• The file server has been running into some scaling issues which is resulting in timeouts on uploads and downloads
  • We've made a couple of tweaks to some server settings which may alleviate these issues in the short term, but are also looking into longer term solutions as well (don't have any timelines on this at the moment unfortunately)

from session-ios.

Thanks @mpretty-cyro . Let me know if you need anything further.

from session-ios.