notalib / lyt Goto Github PK

This is a re-write of issue #6

Calling issueContent never fails, even when it should. If called with a non-existant ID, issueContent responds with

<issueContentResult>true</issueContentResult>

even though the ID doesn't exist. The correct response, I believe, is a parameterFault

Since issueContent accepts anything, the non-existant item is also added to the user's list of issued books. So when getContentList is called, it contains an item marked "unknown", which it shouldn't.

Note that it's pretty easy for a user to accidentally request that a non-existant ID is issued. For instance, simply mis-typing a URL that contains a book-ID can result in that (wrong) ID being "issued" even though it shouldn't. So it's a bug that it's easy to run into.

The RPC to /getContentList returns a list containing items "unknown" for non-existent IDs. These items should not be in the list at all.

I'm assuming the problem lies in /issueContent, which - I believe - adds IDs to the content list, regardless of whether those IDs exist or not. Either the invalid IDs shouldn't be added to begin with, or they should be not be removed from the /getContentList-response before it's sent. Eitherway, it should be done server-side.

Implement so they shift back and forward in time with increasing speed as you hold the buttons.

When retrieving a user's bookshelf (i.e. calling /getContentList for issued items), the response XML contains a hack in order to include both a book's title and author, by using "$" as a separator.

Specifically, a contentItem node looks like this

<contentItem id="some-book-id">
    <label xml:lang="en">
        <text>Book title$Author name</text>
    </label>
</contentItem>

For one, this is probably at odds with the Daisy specification. It's at least against the spirit of it, I believe.

Second, it's inflexible. If more data needs to be sent, the client-side would need to be updated to "know" what data the different sections of the string contain.

But most importantly, it's fragile. If a book's title – or for that matter the author's name – contains a "$", it becomes impossible to parse the text correctly.
And it's not a far-fetched risk. Imagine an eponymous autobiography by the singer Ke$ha, for instance: Ke$ha$Ke$ha (I know, it's a terrible thought, and not just for technical reasons). There's just no good way to parse a string like that.

Possible solutions

A better solution would be to include an attribute node in a different namespace:

<s:Body ... xmlns:nsx="http://www.nota.nu/xmlns">
...
<contentItem id="some-book-id" nsx:author="Author name">
    <label xml:lang="en">
        <text>Book title</text>
    </label>
</contentItem>

This, as far as I know, would not conflict with any specification in any way, and would be straight-forward for the client to parse.

Alternatively, a separate node might be included, although I don't know if the Daisy spec would frown on this:

<contentItem id="some-book-id">
    <label xml:lang="en">
        <text>Book title</text>
    </label>
    <nsx:author>Author name</nsx:author>
</contentItem>

Either of these XML-based solutions would be infinitely more robust and flexible than the $-trick.

To quote Nokogiri:

XML is like violence - if it doesn’t solve your problems, you are not using enough of it.

A third option – which would still be against the Daisy spec, but not more against it than the $-trick already is – would be to use the <text> node for JSON-data:

<contentItem id="some-book-id">
    <label xml:lang="en">
        <text>{"title": "Book title", "author": "Author name"}</text>
    </label>
</contentItem>

Again, this would be straight-forward for the client to parse, and infinitely flexible.

And Ke$ha can't ruin it.

Autocomplete event.data is undefined.

If you call getContentList with the firstItem parameter set to anything other than 0, an empty list is returned.

Example:

Request (just the SOAP body):

<ns1:getContentList>
    <ns1:id>issued</ns1:id>
    <ns1:firstItem>5</ns1:firstItem>
    <ns1:lastItem>10</ns1:lastItem>
</ns1:getContentList>

Note the non-zero firstItem

Expected response: 6 items, from index 5 to 10 (inclusive)

Actual response:

<contentList totalItems="0" firstItem="0" lastItem="-1" id="issued">
    <label xml:lang="en" dir="">
        <text>issued</text>
    </label>
</contentList>

Note that the firstItem and lastItem attributes do not match the ones requested. In fact, had the request been for items 0 to -1, there should be more than 20 items in the list. So:

1. The response reflects wrong values for firstItem and lastItem
2. The response's content doesn't match the wrong firstItem/lastItem values either

A range of books get the following error/warning:

Section: Failed to find media for offset 0
utils.js:12Player: failed to get media

I've only identified three, but guess it's a common fault to do with media offset somehow

http://m.e17.dk/dev/#book-play?book=36819
http://m.e17.dk/dev/#book-play?book=36763
http://m.e17.dk/dev/#book-play?book=19687

... and try not to break Chrome in the process

LYT.search.full('myterm')
Throws ReferenceError: request is not defined

Maybe I am using it wrong?

I renamed this issue from disabling interface (solved by decreasing opacity while loading) - now it's only concerning when pageloading starts and ends.

I think we should be as conservative as possible when starting/ending the pageloading icon- so for instance the pageloading on bookplay (by far the most important btw) should start immediately when the bookshelf or book-details page is clicked and wait until everything is ready including the page text until it stops loading - otherwise clicking play would actually not work/be confusing

I.e. MIT, GPL, LGPL, Apache or similar...

If we want the permalinks to not contain the somewhat strange section ids but instead the number the section has in the book we should make this number available on the section object so we can link correctly to playback of specific sections.

Alternatively if we choose to use the id's we need a way to get a specifc section object by its id.

I still can't figure out whether we missed a commit or something - but we are back at the accumulating transitions between search page and book-details - rather comic interface design - solution is stopimmediatepropagation i guess

Log off and remember login option

It would be useful if you could rewind to the previously played "segment", i.e. the previous paragraph or comic panel (not the previous chapter or section). Especially for short paragraphs or comicbook panels with little text but a lot of art, it'd be nice to be able to go back and re-read/re-view/re-listen, since the next segment will be shown very quickly.

(It'd be nice to be able to skip forward too, but I imagine that's trickier, since that next part of the mp3 would have to have been loaded already)

(this issue supersedes issue #3 and issue #9)

The playback object should provide data-model suited for audio playback

Once in a while mediaFor hangs for a long while (3 - 5 minutes I think) getting a new section then suddenly returns a long list of 404 errors on mp3's it didn't find. Send you a screenshot.

see /dev/design/mine_boger.png

There's basically two sorts of errors: incidents where we want to let the user know, and incidents where we want to log the error and go on.

Let the user know

Let's break it up: rpc errors, file parsing errors and streaming errors.

RPC errors should generally be handled the same way that login errors are handled right now: try again - say three times, otherwise let the user know: It looks like we can't playback the audiobook right now. Please try reloading the page. If the problem persists, check your internet connection or try again later. If the problem persists don't hesitate to contact us: support phonenbr.

File parsing errors occur when - for some reason - we are not able to parse - say - an ncc correctly. The easiest way to check that is in fact to check if there are any items in the index - if not, some form of error occurred. Considering the state of the catalogue, there may be millions of reasons why this could happen. So we need the book-id to debug. Send a mail to the maintainer (Lars, for now) and let the user know: there was a problem parsing this book. The problem has been reported. Please try again later. If the problem persists, don't hesitate to contact us.

Streaming errors - probably occur because of either lost connection or some logoff in the background (i.e. if the user switch IP). Try logging in and continue playback. If the connection is lost (is there any solid way to check that?), let the user know: It looks like you've lost your connection to the Internet. Please try reloading the page, or try again later.

Log the error

If there are interface errors or errors concerning parsing of books or files that don't affect playback, we want to know - but we don't want the user to know. This may apply, for instance, to the warnings we send from render and parsing methods. I think email is an easy to implement sollution - we get the error and can check it out right away.

(Edit: Removed Lars' email-address... this is a public site after all)

The top item of the bookshelf needs a "now playing" icon and should only react to click by showing the book-play page. The finished icons are not ready yet, but we can use images/nowplaying.png

We can not be totally sure the currently playing book is always on top of the bookshelf list, i.e. it's a coming feature that books can be added to the shelve without playing them - which would result in them being on top of the list.

I suggest we save the currently playing booknumber in settings and put it on top of the list manually. The only action that can change currently playing, is clicking play on a new book.

Book details page should be able to do two things: play book directly and put book on shelve. Right now the "play" link doesn't always work - and if it does, pageloading is not shown while the book loads.

The other function - put on shelve - should just call issuecontent and make a transition to the bookshelve.

Some pages do the pagetransition before the actual work gets done - especially book-play and book details - ending up in the user waiting for a book to change without knowing. There's two ways to go about it - either to erase the existing text values (so you don't see i.e. Harry Potter while you wait for Thomas Mann) and show the pageloading message - or maybe alternatively do the work in pagebeforecreate - and delay the transition until the page is finished loading the book

Can i get a logged off event from service to trigger af redirect to the login page.

Various error constants are currently in the global (i.e. window) namespace. To keep the code's global footprint to a minimum, these constants should probably be moved to their own namespace. E.g. move window.RPC_TIMEOUT_ERROR to LYT.constants.RPC_TIMEOUT_ERROR or simply LYT.RPC_TIMEOUT_ERROR

Though it's nice to have a central config-file (it's something I advocated myself), it's become a little cumbersome, and the file itself has gotten somewhat complicated. And it can never really be complete anyway, since some options don't belong there, since they're not constant.

Perhaps it's better to break apart config.coffee and move the options to the files where they matter?

This issue is a little hard to debug if we don't get the terms right: As far as I can see, there's three different ways the rewind/forward buttons could react:

1 By chapter
2 By section
3 By subsection

Right?

I think the most generic way would be to implement it as a setting that defaults to section (i.e Opikayana ?book=36738 would be played page by page). The other jumps could either be triggered by manually changing default settings, implement it as user-prefs or adding interface buttons.

Right now - I think - the interface tries to switch per subsection - it doesn't always work out - try Opikayana for example. Actually I think subsections are the least important option - so we could choose not to implement that and just leave it as an unhandled setting. In my view the most obvious is to switch between sections. But chapter switching is already implemented, so we might as well include it as a setting option.

Implement cartoon mode where you can shuffle back and forward between illustrations with autoplay of the next illustration disabled.

When searching or looking for search suggestions (autocomplete), the server returns a lot of data, much of which is pointless and/or duplicated.

Both search and autocomplete responses returns the same JSON structure, but it's used in different ways. And in the case of autocomplete, it presents a huge data overhead.

Example autocomplete response:

{
    d: [
        {
            "__type":       "Nota.Lyt.Item",
            "totalcount":   null,
            "index":        null,
            "author":       null,
            "title":        "T",
            "series":       null,
            "seqno":        null,
            "totalcnt":     null,
            "media":        null,
            "teaser":       null,
            "keywords":     "Some book title",
            "imageid":      null,
            "speaker":      null,
            "resultstatus": "OK"
        },
        ... more results here
    ]
}

The only values that are useful are keywords and resultstatus. But resultstatus is needlessly repeated for each result, although it pertains to the entire result set.

The response could be streamlined to simply:

{
    results: [
        "Some book title",
        "Some other book title",
        ... more titles here
    ]
}

This would require much less processing on the client, and much less data transfer. Both of which are obviously very desirable on a mobile device.

The resultstatus value can be left out entirely. Any special status values can be expressed in the response's HTTP status. I.e. if the server responds with 200 OK the search can be assumed to have worked fine.
In the case of a search that returns no results, the resultstatus value is set to "NORESULTS", although returning an empty results array would communicate exactly the same. Alternatively, the server could in that particular case respond with 404 Not Found.

All in all, the response data can be reduced by 80% with no loss of information.

When doing a "full" search, the server returns its results using the same structure, however it's more fleshed out.

Here's an example response:

{
    d: [
        {
            "__type":       "Nota.Lyt.Item",
            "totalcount":   "8",
            "index":        "7",
            "author":       "Some author",
            "title":        "Some book title",
            "series":       "SID0000000000",
            "seqno":        "1",
            "totalcnt":     "7",
            "media":        "XX,YY,ZZ",
            "teaser":       "This book is about etc etc...",
            "keywords":     "keyword1 ; keyword2 ; keyword3 ",
            "imageid":      "00000",
            "speaker":      null,
            "resultstatus": "OK"
        },
        ... more results here
    ]
}

Again, a lot of information is duplicated across each result, causing a lot of data overhead.
As before, the resultstatus value can be left out entirely, since it can be expressed via the HTTP status, or is simply self-evident from the result set.
Also evident from the set, are the totalcount and index values. Since the results are returned as an array, those values are given anyway.

Oddly, the results do not actually (explicitly) return the ID of the book, which is by far the most important piece of information. Instead it returns imageid which contains the book's ID, but isn't labelled as such. Why not call it id? Even if it's the same value, I see no reason to label it as the image ID.

A lot of other values are not used by the client at present. But even leaving those values in place, the response can still be reduced to:

{
    results: [
        {
            "id":           "00000",
            "author":       "Some author",
            "title":        "Some book title",
            "series":       "SID0000000000",
            "seqno":        "1",
            "totalcnt":     "7",
            "media":        "XX,YY,ZZ",
            "teaser":       "This book is about etc etc...",
            "keywords":     "keyword1 ; keyword2 ; keyword3 ",
            "speaker":      null
        },
        ... more results here
    ]
}

Here, the response is reduced by about 20% with no loss of information.

There is a very annoying bug in chrome/mac firefox/mac firefox/pc that randomly disables books from being played. In Chromeo it's always concerning the .html file in the Daisy book, and the logging always stalls on:

DTB: Getting: http://m.e17.dk:80/DodpDistributor/Distribute.aspx?session=100017&book=36738&file=//LOCAL/UNI0/367__/36738/36738.htm

In Firefox the files that fails to load are random - but the error is the same - except the fileload sometimes don't get started. Funny thing is when the page is reloaded, it almost always works!

So this problem has to do with multiple ajax calls and probably closing connections

So...

At times there are some warnings from the playlist object that may influence playback/nAn etc. To reproduce, choose "Det stille angreb" and continue playback until first section shift:

Section: Failed to find media for offset 4.440816402435303
utils.js:26Player: failed to get media segment for offset 4.440816402435303

Well by closer look, this happens all the time - intentionally maybe?

To reproduce, choose "Kontrakten" and play - watch the log

Section: Failed to find media for offset -0.03783644363284111
utils.js:26Player: failed to get media segment for offset -0.03783644363284111

Get the SERVICE_MUST_LOGON_ERROR event working so you can listen and handle it from one place.

Allright, so here's the schedule for how we deal with this bug for the first release.

1. It's a bug in IOS, confirmed - others than us spending more time with this stuff has the excact same issue. Therefor we don't try to solve it, but take steps to minimize the usability issues
2. The bug only appears when a new mp3 file is fetched from the server. Therefore it's (even more) annoying that every book starts with numerous (strictly irrelevant) paragraphs of Daisy metadata. I'll make a new issue that describes which these are (names like dc:something) and suggest they are removed to the ending of the book. This step will make the bug a lot less dramatic for most books.
3. We try to fetch the file duration 10 times in a loop, while we keep the user waiting (don't forget the loading icon)
4. If that doesn't succeed, we start playing the file (without duration) and set a flag: IOSBUGACTIVE or something. When the user hits the back forward buttons or timeline, we make en alert that explains the problem - and keep playing.

Submitted this as a bug at jPlayer waiting for a reply.