The BoM for the Procurer lists all required minerals two times with different amounts. Doesn't happen with all blueprints.
This happens for example with the Augoror and the reason is the changed BoM since Retribution. The old algorithm to determine if it's "advanced" or "non-advanced" manufacturing doesn't work anymore.

The readme is pretty minimalistic compared to what is going on in the code. A section describing the structure of the service should be added.

We could generate and display some stats from ESI market's data store such as:

• Most-updated orders for detecting unusual activity patterns/bots
• Order 'velocity' per region (opened, closed, updated...) for esimating general market activity
• Total number of orders tracked for seeing global fluctuations in activity

Any more ideas?

Get to work, fools!

Now that we have switched to gRPC for our cluster's internal communication we can generate OpenAPI docs by using grpc-gateway on which jumpgate is based. The individual service's Protobuf annotations are there, we just need to combine the individual generated Swagger docs using something like swagger-combine and package them up for deployment with swagger-ui. We may even be able to use parts of esi-swagger-ui so our docs would look like ESI's documentation.

Whenever esi-markets detects an update of an order within a market in a region it saves the matching combination of regionID and typeID. At the moment this mechanism is only used for logging. This describes an endpoint for streaming these combinations as updates occur. Clients like vue43 could then update their local view when receiving such a region/type pair resulting in 'real-time' display of market data. Also, more advanced market statistics could be reactively generated in downstream services such as market-stats (95th percentile...). We could even use that change detection mechanism for implementing an order watching and notification service.

There already is dummy code in esi-markets for this. The 'source' stream could be added to the store or its external interface could be used. Given the fact that the gRPC implementation already has access to the store for querying orders it would be easy to add the source stream to the store instance, too.

Would be great to see the most benefitial item for every LP Store.

http://i.imgur.com/bgHBTNH.png

Looks like something Bootstrap-related may be off. Notice the square placeholder character at the left of the alert div.

Hi,
When i try to build the docker image i get this error:

ERROR in /vue43/components/Search.vue.ts
[tsl] ERROR in /vue43/components/Search.vue.ts(63,7)
      TS2322: Type 'string | (string | null)[]' is not assignable to type 'string'.
  Type '(string | null)[]' is not assignable to type 'string'.

ERROR in ./node_modules/css-loader??ref--6-1!./node_modules/postcss-loader/lib??ref--6-2!./node_modules/stylus-loader??ref--6-3!./stylus/main.styl
Module build failed: Error: /vue43/node_modules/vuetify/src/stylus/components/_calendar-daily.styl:5:40
   1| @import '../bootstrap'
   2| @import '../theme'
   3|
   4| v-calendar-daily($material)
   5|   background-color: $material.calendar.background-color
---------------------------------------------^
   6|   .v-calendar-daily__intervals-head
   7|     border-right: $material.calendar.line-color 1px solid;
   8|   .v-calendar-daily_head-day

$material.calendar has no property .background-color
    at v-calendar-daily() (/vue43/node_modules/vuetify/src/stylus/components/_calendar-daily.styl:5:1)
    at ".theme--dark.v-calendar-daily" (/vue43/node_modules/vuetify/src/stylus/theme.styl:13:3)
    at dark() (/vue43/node_modules/vuetify/src/stylus/theme.styl:12:1)
    at theme() (/vue43/node_modules/vuetify/src/stylus/theme.styl:4:1)

    at CachedPathEvaluator.Evaluator.visitMember (/vue43/node_modules/stylus/lib/visitor/evaluator.js:351:11)
    at CachedPathEvaluator.Visitor.visit (/vue43/node_modules/stylus/lib/visitor/index.js:28:40)
    at CachedPathEvaluator.Evaluator.visit (/vue43/node_modules/stylus/lib/visitor/evaluator.js:160:18)
    at CachedPathEvaluator.Evaluator.visitExpression (/vue43/node_modules/stylus/lib/visitor/evaluator.js:644:26)
    at CachedPathEvaluator.Visitor.visit (/vue43/node_modules/stylus/lib/visitor/index.js:28:40)
    at CachedPathEvaluator.Evaluator.visit (/vue43/node_modules/stylus/lib/visitor/evaluator.js:160:18)
    at CachedPathEvaluator.Evaluator.visitExpression (/vue43/node_modules/stylus/lib/visitor/evaluator.js:644:26)
    at CachedPathEvaluator.Visitor.visit (/vue43/node_modules/stylus/lib/visitor/index.js:28:40)
    at CachedPathEvaluator.Evaluator.visit (/vue43/node_modules/stylus/lib/visitor/evaluator.js:160:18)
    at CachedPathEvaluator.Evaluator.visitProperty (/vue43/node_modules/stylus/lib/visitor/evaluator.js:687:22)
    at CachedPathEvaluator.Visitor.visit (/vue43/node_modules/stylus/lib/visitor/index.js:28:40)
    at CachedPathEvaluator.Evaluator.visit (/vue43/node_modules/stylus/lib/visitor/evaluator.js:160:18)
    at CachedPathEvaluator.Evaluator.visitBlock (/vue43/node_modules/stylus/lib/visitor/evaluator.js:720:39)
    at CachedPathEvaluator.Visitor.visit (/vue43/node_modules/stylus/lib/visitor/index.js:28:40)
    at CachedPathEvaluator.Evaluator.visit (/vue43/node_modules/stylus/lib/visitor/evaluator.js:160:18)
    at CachedPathEvaluator.Evaluator.invoke (/vue43/node_modules/stylus/lib/visitor/evaluator.js:1087:17)
 @ ./stylus/main.styl 4:14-187
 @ ./store/index.ts
 @ ./store ^\.\/(?!-)[^.]+\.(js|ts)$
 @ ./.nuxt/store.js
 @ ./.nuxt/index.js
 @ ./.nuxt/client.js

 ERROR

  Error: Webpack build exited with errors

I try to run the services using docker-compose. Here is my docker-compose.yml:

version : "3.7"

services:  
  vue:
    build: https://github.com/EVE-Tools/vue43.git
    container_name: vue43
    restart: unless-stopped

  esi-markets:
    build: https://github.com/EVE-Tools/esi-markets.git
    container_name: esi-markets
    environment:
      - REFRESH_TOKEN=
      - CLIENT_ID=
      - SECRET_KEY=
      - GRPC_HOST=43000
    networks:
      - gateway
    restart: unless-stopped

  static-data:
    build: https://github.com/EVE-Tools/static-data.git
    container_name: static-data
    environment:
      - LOG_LEVEL=info
      - PORT=43000
      - DB_PATH=static-data.db
      - ESI_HOST=esi.tech.ccp.is
      - STRUCTURE_HUNT_HOST=stop.hammerti.me.uk
      - DISABLE_TLS=false
    networks:
      - gateway
    restart: unless-stopped

  top-stations:
    build: https://github.com/EVE-Tools/top-stations.git
    container_name: top-stations
    environment:
      - CRON=@hourly
      - DB_PATH=/data/top-stations.db
      - LOG_LEVEL=info
      - ESI_MARKETS_HOST=esi-markets:43000
      - PORT=43000
    networks:
      - gateway
    restart: unless-stopped

  market-stats:
    build: https://github.com/EVE-Tools/market-stats.git
    container_name: market-stats
    environment:
      - SEED_DB=false
      - CRON=0 52 * * *
      - LOG_LEVEL=info
      - POSTGRES_URL=postgres://market-stats@localhost:5432/market-stats?sslmode=disable
      - PORT=43000
    networks:
      - gateway
    restart: unless-stopped

  jumpgate:
    build: https://github.com/EVE-Tools/jumpgate.git
    container_name: jumpgate
    environment:
      - LOG_LEVEL=
      - LISTEN_HOST=:8000
      - ESI_MARKETS_HOST=esi-markets:43000
      - MARKET_STATS_HOST=market-stats:43000
      - STATIC_DATA_HOST=static-data:43000
      - TOP_STATIONS_HOST=top-stations:43000
    ports:
      - 8000:8000
    restart: unless-stopped

networks:
  gateway:

Just testing hooks. Ignore this.

The table 'api_skill' and 'api_skillgroup' are empty and this generate 'violates foreign key constraint' if I insert an new api key.

If someone is interested in a deep-dive into memory allocation internals, this one is for you. Statically built versions of esi-markets tend to use way more memory that their dynamically linked counterparts. The repository contains instructions on how to build a static version of esi-markets which can be used for profiling etc.. How can we get a statically-linked version which uses less memory at runtime?

We should add Prometheus metrics collection to jumpgate so we can analyze the system's usage and performance.

At the moment the Bill of Materials (BoM) only show the raw matierals (also called commodity or resource) and/or highest tier components (items that can be manufactured) required to manufacture the item. When the BoM contains components it is not possible to see their BoM.

The manufacturing tool should therefor allow the user to display the BoM of all components that are required by the blueprint.

Some sort of tree view should be the best way to implement this. The user should be able to decide which components BoM he wants added to the overall BoM.

I'm trying to run e43 on my VPS which has 2 GB memory (~1,7 free). While processing the 'ramAssemblyLineTypeDetailPerGroup' table the script stops with the following error.

django.db.transaction.TransactionManagementError: Transaction managed block ended with pending COMMIT/ROLLBACK

I also had some other errors which I can't remember yet, but the fact, that this script is using my whole memory should be fixed.

Vue43 depends on many NPM packages. It would be great if we could update them automagically by using something like greenkeeper.

In order to speed up initial loading time for a given type's regional view, its loading should be deferred via AJAX like the region tab in the type view

In cases where the EVE API is unavailable and thus slowing down the requests or when there are not enough worker processes available to handle all the scheduled tasks, the task queue will keep growing, because Celery will automatically add duplicate tasks.

This behavior should be prevented.

Need to create the models to handle proper handling of corporate data in the system. Specifically should use character rights from API but be manageable by the CEO/director. This will require a new front-end.

Add the ability to filter Journal Entries by refType.

Automatically use HTTPS for login/auth and wallet stuff. Also serve assets via HTTPS.

At the moment it is a ~1kloc blob of Go code in a single file which needs to be split up into individual packages in preparation for reactive stat generation made possible by #46.

Right now the performance in the "Browse Market" view is not great. The market groups are loaded dynamically on every request.

To improve performance all market groups could be "pre-compiled" in a simple JSON file. The format of the JSON file will be provided by @zweizeichen. To create the JSON I propose a Django command that can be run from the command line.

The only drawback to that solution is that the file has to be "compiled" everytime the database dump is imported.

Update 1

@gtaylor proposed to generate the file in the view and just cache it or have it generated periodically with a celery task. Either way, this would avoid the drawback described before.

After you enter your API ID and Verification ID you can select the characters you want to import. Hitting the "Add Character" button results in an Django exception:

Environment:

Request Method: POST
Request URL: http://element-43.com/settings/api/character/*/*/

Django Version: 1.4
Python Version: 2.7.3
Installed Applications:
('django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.sites',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.admin',
 'django.contrib.admindocs',
 'django.contrib.humanize',
 'south',
 'devserver',
 'apps.market_data',
 'apps.api',
 'eve_db')
Installed Middleware:
('django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware')


Traceback:
File "/opt/python27/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  111.                         response = callback(request, *callback_args, **callback_kwargs)
File "/opt/python27/lib/python2.7/site-packages/django/contrib/auth/decorators.py" in _wrapped_view
20.                 return view_func(request, *args, **kwargs)
File "/home/app/element43/webapp/element43/apps/market_data/views/settings.py" in api_character
  159.                  if len(key_info.key.expires) == 0:

Exception Type: TypeError at     /settings/api/character/*/*/
Exception Value: object of type 'int' has no len()

It seems like expenses are not taken into account.

Go here http://www.element-43.com/market/29668 and click between Orders and Regions tab. The page length extends in Orders to the point of needing a vertical scroller, whereas Regions doesn't.

Probably best to always show the scrollbars to avoid the horizontal bouncing.

My complaints related to the search bar:

• The item icons are circular - It may look cool, but it cuts off the tech level indicators.
• The search results aren't proper links - As a result, there is no way to open the item page in a new tab, making viewing the details of multiple items way more complicated than it should be.
• On losing focus, the search query will be cleared - I don't find this to be useful behaviour, and to make things more confusing, the results of the last search will remain shown once the search bar regains focus.
• Also, having the search bar autofocus on page load would be pretty useful.
• Having the search bar available in the toolbar on every page would be nice.

The entire page is being cached (including the username at the top right) - we should implement partial caching.

Initial install seemed to go fine but no joy at the very end, no price data to DB as near as I can tell. Maybe this will help explain:

http://pastebin.com/a7hbcrge

I wanted clean logs and such so I restarted the VM and now I can't figure out how to connect with supervisorctl at all. Do I need to source into a virtualenv or something? Lost.

gtaylor said you've got finals, good luck with those and THANKS

While running our own CI system can be fun, it is yet another service to maintain. We could migrate to one of the numerous CI platforms which offer free services for open-source projects.

Same as in the ask/bid view - we are experiencing extreme performance issues here, too.

I often need to make multiple searches. When I search for an Item, it takes me to a new page, with no way to search for another item without clicking back or opening a new Element-43 tab. It would be nice to have a search bar at the top of each items page that opens in a new tab so we could do multiple searches quickly.

I'd suggest prioritizing PLEX in the ajax search when entering 'plex'. Right now you get stuff like Complex Organisms and Complex Reactor Blueprint, but not Pilot License Extension (which is what most people are probably expecting when they enter 'plex').

It looks like something's an issue with the responsive design — what you get is exactly what you get on the desktop when window width is < 768 px. Rotating to landscape has no effect even though my display should have enough pixels to trigger the next breakpoint.

Galaxy Note II, Cyanogenmod 10.1.3 (Android 4.2.2)

The flate2 crate used for compressing esi-market's persistence is outdated. It should be upgraded to the latest version and some simple tests should be performed to pick the fastest variant (miniz_oxide vs. miniz etc.).

Related code:
https://github.com/EVE-Tools/esi-markets/blob/6b135fb08b5a421d7885663e285d60214756abee/Cargo.toml#L13-L14

https://github.com/EVE-Tools/esi-markets/blob/6b135fb08b5a421d7885663e285d60214756abee/src/store/mod.rs#L107-L163

https://github.com/EVE-Tools/esi-markets/blob/6b135fb08b5a421d7885663e285d60214756abee/src/universe/mod.rs#L198-L244

Task apps.api.tasks.ProcessResearch[*] raised exception: AttributeError('reserachStartDate',)

api/tasks.py:
start_date=pytz.utc.localize(datetime.datetime.utcfromtimestamp(job.reserachStartDate)),

probably a typo - will be fixed shortly.

We're using CREST for generating our sitemap. This is bad and we should/have to move to ESI before CREST gets taken offline this spring. The way ESI is designed we need to do a little more work for determining market types but it is doable.

Right now we can only view the profit we made with a certain type over the complete timespan of our data. An ability to view the profits of e.g. the last 7 days should be useful.

It seems like the quicklook view tends to be confusing - it should be easier to get an instant impression of a given type's prices with a redesigned view.

It takes the page about 30 seconds to load. Performance has to be increased drastically.

At the moment there is no real documentation on how to get started with development. Also a simple to set up dev environment is missing. We need documentation detailing a convenient dev workflow on:

• How to modify the frontend
• How to modify existing services
• How to add a new service

While modifications to the frontend can be done locally by simply using the production API (and disabling CORS in the browser), developing new services or modifying existing ones requires a local instance of the backend (if the service needs to access other services in the cluster). We're using Kubernetes in production so it would only be fitting to use that for our dev environment, too. Individual YAML manifests for each service could be deployed to a local minikube instance or an arbitrary Kubernetes cluster for development while telepresence is used for proxying traffic of the process on the devs machine into the cluster.

Some form elements like switches are not perfectly aligned to the right in the quicklook or the market table view. It should be a matter of some simple CSS fixes either in the components or the addition of a global class to correct this issue.

It would be pretty rad if you could go from an item detail page like this: http://www.element-43.com/market/17363/

To a trade finder to pre-configured for the item in question. This could be a pretty neat thing if done correctly. The current tradefinder requires you to enter a start/end region, which is appropriate for the more general case.

For a specific item, perhaps we do a bunch of cross-region comparing and find the best route. They'd still need to filter out some obviously shite routes, and we may need to give them some additional settings to tweak what gets returned (perhaps we're not interested in low/nullsec routes).

When a user updates a price for a material the price should be saved in the session or maybe in the browser (html5).

This way a user doesn't need to re-enter prices if he selects another blueprint or just reloads the page.

The price should be saved as soon as the user leaves the field.

Absolute minimum tables needed from static data dump in the short term:

invTypes
mapRegions
mapSolarSystems
invMetaGroups
invMarketGroups
staStations

That should be enough to get us rolling.

Show the perfect ME an PE next to the corresponding input fields in the manufacturing job form.

At the moment there is no retry mechanism for the first request (for getting X-Pages) when fetching orders from ESI for a region/structure. If that request fails, the whole region won't get an update. A retry mechanism should be added here and here. Code can be copied from here. However if we want to do this The Right Way™ we should perform a randomized exponential backoff in those cases.

The order overview should allow you to compare your ask/bid orders which are on the same station to support station traders.

User registers, gets the link in the email and clicks on it, gets a 404 page from the website and user is not activated.

I just manually activated a user because of it. Also I believe barbers had the same problem.