MTNI-95 ⁃ Changing urls without loading new page (web apps) fails to fetch new page data. about memex HOT 14 CLOSED

When I look into this issue, then I found that this is due to the lodash/debounce time to callback.
When I reduce the time to the 100ms then it works well but the title is not coming from that web page, it is from the previous pages.

from memex.

It looks like this

from memex.

Thanks for looking into this @mukeshkharita
Seems like tinkering on the current solution to detect URL changes does not work then.
Maybe we need a completely different approach to detect URL changes in web apps.

In the call we talked about this here: https://developer.mozilla.org/en/docs/Web/API/History_API
Have you tried it with that?

from memex.

In the call we talked about this here: https://developer.mozilla.org/en/docs/Web/API/History_API
Have you tried it with that?

This is the HTML5 history manipulation API. It's not really appropriate to solve this. It was pointed out in the call as more of a suggestion as this is generally the API single page web apps use to change the location/URL state without needing the page to be reloaded (routing packages like react-router would use this).

The event that's being listened on in the extension is the tabs.onUpdated event from web extension API. It might be worth looking to confirm that this event gets fired on all URL state changes via the history API, not just location state changes via more traditional means (<a href=""> for example). If it doesn't fire on all state updates, might want to look for work-arounds, else we need to think of some other way to listen for location changes.

from memex.

I think there is only one problem and that is It is not taking the correct title of web page so that we could XMLHttpRequest for particular page and can save it.
@oliversauter @poltak could you suggest me that how could I proceed with this issue.
Always welcome of suggestions.

from memex.

From the top of my head I don't know any ways.
Did a some research, you may find valuable pointers in these threads:
https://stackoverflow.com/questions/4570093/how-to-get-notified-about-changes-of-the-history-via-history-pushstate/25673946#25673946

Does the tabs.onUpdated Listener detect the change when switching urls in tabs in general and just fails to execute?
If yes, another idea: Could you listen to the changeInfo object to change?
https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/tabs/onUpdated

or maybe stopping to redo the debouncing has something to do with it?
https://github.com/WorldBrain/WebMemex/blob/master/src/activity-logger/background/index.js#L19

from memex.

Regarding the XMLHTTP request, seems like a dirty way to do it here, without having to worry about deduplication. It would create 2 visit items and 2 page objects then, right?

from memex.

As currently we are using the tabs.onUpdated.
We are getting all the switching urls but the problem is

1. status is loading so that it is not able to give the correct title of that while we are getting the correct snapshot.
2. We cannot search because because we don't have any title to this page

Yes that's right we shouldn't use the XMLHTTP request.

from memex.

Yes, no reason to do an XHR; this data is readily available in the tab.

@mukeshkharita What do you mean by this:

1. status is loading so that it is not able to give the correct title of that while we are getting the correct snapshot.

What is "status" here and how are you determining that it is loading? Or: how does this statement relate to the code?

Have you tried putting simple console.log calls or debug points to see what is actually happening? You should have some expected behaviour that you can compare with the actual behaviour, which you can then form a question to investigate further.

from memex.

Yes, actually I did console.log and I found that the title of page is same sa previous one(I mean the webapp first page with loading). and one thing is also there, the tabId is same.

from memex.

As just discussed in Slack:

To get the content from a page you can do: 'document.body.innerText' and 'document.title' in the console.

Important is that we can detect the url change in a web-app, if it does not reload its page.
This is the reason why it works on twitter, where you reload the whole page when changing the url.
However on Asana it does not work because it does not reload the page.

It might be a way to solve this by playing around with how the 'tabs.onUpdated' listener works, that is here: https://github.com/WorldBrain/WebMemex/blob/master/src/activity-logger/background/index.js#L16

Documentation for that you find here: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/tabs/onUpdated

from memex.

Thanks @oliversauter, I will let you know about status of current issue and will try to solve it asap.

from memex.

To get the content from a page you can do: 'document.body.innerText' and 'document.title' in the console.

Sadly, this stuff is done via the background script and thus doesn't have access to document of a given page; it must go through the tabs API, or messaging API to allow the scripts to pass data, however:

Yes, actually I did console.log and I found that the title of page is same sa previous one

Which seems to indicate the browser.tabs.onUpdated event is being fired but the actual tabs state isn't being updated? The tabId being the same is expected.

@mukeshkharita maybe you could try doing a browser.tabs.get inside the browser.tabs.onUpdated event handler and see what state that gives you (compare to the state given to the event handler).

EDIT: There is also the Web API event WindowEventHandlers.onpopstate which gets fired on changes to window.history (same API talked about earlier; allows the app to change location without page reload). This is a pretty "non-webextension" way of doing it though, but it may be useful to see what browser.tabs.get would be return on these events, if nothing else is working.

from memex.

somehow this works again :) We tried it on 2 different computers.

from memex.