vozhyk- / megahttp Goto Github PK

Should have a reference to the megaApi it needs to use (global or account).

http://localhost:8080/status/

Show some status information:

• Current downloads
• Memory usage
• Accounts logged into
• Current configuration

http://localhost:8080/[email protected]/path/to/file

Needs to detect the type from buffer.

So that lines from different threads is not mixed together.

Use locks / condition variables / ...

Currently, megahttp allocates memory for the whole file. This will result in bad things happening when trying to download files too big to allocate.

What needs to be done is:

• separate buffer functionality from file_cache_item into file_buffer
• split all files into blocks
  • use a block_size parameter
  • determine which block the requested range belongs to in file_buffer when chunk requested or stored
• rename file_cache_item -> cached_file
• change file_buffer structure to:
  • (from buffer = vector<char>)
  • buffer = map(chunk_number -> file_buffer::block)
  • change cached_file.get_chunk to copy data instead of returning its address
    • rename to get_data
  • levels of abstraction:
    • GC should manipulate blocks
    • downloads / responses should manipulate ranges
• garbage collector should be called on block allocation instead of file allocation
• garbage collector should search all blocks
  • some blocks should be protected from deletion
    • protect [response_callback.position .. response_callback.download.position) (∀ response_callbacks serving the file)
      • every response_callback should have its own download
        • downloads should start MEGA downloads for missing pieces and skip pieces already in cache
          • cached_files should track which ranges are downloaded and which are protected from GC
            • make a range class
        • pause downloads which have gone too far from the response (so that they don't prevent a lot of memory to be released and don't download unnecessarily)
  • (optimization for after this bug) sort blocks (to be deleted) from unused files from tail of file to head of file (instead of least-recently-used first)
• empty (no chunks downloaded/allocated in buffer) unused cached_files should be deleted

WIP on branch big_files

http://localhost:8080/...?docache=no or similar.

file_offset is not reset on new request

On Raspberry Pi (B), Buildroot 2016.11 http://server_ip:8080/status reports very small values of total memory used by the cache (usually smaller than one of the files in the cache). But it reports the individual files' used memory as designed.
This might be caused by 0cf41ab, but that should be confirmed.

Add log levels, etc.
EDIT: Now make MEGA SDK's own logging behave in the same way

(write in-memory items to disk when memory used by cache > limit)