Need a way to tag release with git tags.

Functionality supporting management of dataset metadata. This means everything but the archive file upload and access control.

REST API
(GET, PUT, POST, DELETE)
/datasets
/datasets/:id
/datasets/:id/submit (The owner of the dataset may submit if dataset is in 'draft' status)
/datasets/:id/approve (The owner of the dataset may approved if dataset is in 'submitted' status and the user is an NGT Administator)
/datasets/:id/unsubmit
/dataset/:id/unapprove

When the REST api raises and error code (500) the UI valdates a POST as successful

Admins and users need the ability to delete Dataset DRAFTs.

Need ability to delete submitted dataset. Only approved datasets cannot be deleted.

Related to #15

The cdiac import will require two new attributes to be added to the Dataset. This will allow the datasets to be searche by these items

• cdiac_import (boolean)
• cdiac_submission_conact (Person)

ngt-archive does not package up ui/static files for distribution.

Can we create many-many mappings between PIs/Contacts and Sites? Some PIs manage multiple sites and some other sites have multiple contacts or PIs (e.g. from various govt agencies)

Created a dataset in the ui, then uploaded a zip file to it. The error below was encountered when trying to download the file.

Internal Server Error: /api/v1/datasets/27/archive/

AttributeError at /api/v1/datasets/27/archive/
'NoneType' object has no attribute 'replace'

Request Method: GET
Request URL: https://ngt-dev.lbl.gov/api/v1/datasets/27/archive/
Django Version: 1.10.4
Python Executable: /usr/bin/uwsgi-core
Python Version: 3.4.3
Python Path: ['.', '', '/var/www/django/ngt-archive/ngt-dev.lbl.gov/lib/python3.4', '/var/www/django/ngt-archive/ngt-dev.lbl.gov/lib/python3.4/plat-x86_64-linux-gnu', '/var/www/django/ngt-archive/ngt-dev.lbl.gov/lib/python3.4/lib-dynload', '/usr/lib/python3.4', '/usr/lib/python3.4/plat-x86_64-linux-gnu', '/var/www/django/ngt-archive/ngt-dev.lbl.gov/lib/python3.4/site-packages']
Server time: Fri, 16 Dec 2016 23:03:55 +0000
Installed Applications:
['ngt_archive',
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'archive_api.apps.ArchiveApiConfig',
'ui.apps.UiConfig',
'rest_framework']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware']

Traceback:

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/lib/python3.4/site-packages/django/core/handlers/exception.py" in inner
39. response = get_response(request)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/lib/python3.4/site-packages/django/core/handlers/base.py" in _legacy_get_response
249. response = self._get_response(request)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/lib/python3.4/site-packages/django/core/handlers/base.py" in _get_response
187. response = self.process_exception_by_middleware(e, request)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/lib/python3.4/site-packages/django/core/handlers/base.py" in _get_response
185. response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/lib/python3.4/site-packages/django/views/decorators/csrf.py" in wrapped_view
58. return view_func(*args, **kwargs)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/lib/python3.4/site-packages/rest_framework/viewsets.py" in view
87. return self.dispatch(request, *args, **kwargs)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/lib/python3.4/site-packages/rest_framework/views.py" in dispatch
466. response = self.handle_exception(exc)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/lib/python3.4/site-packages/rest_framework/views.py" in dispatch
463. response = handler(request, *args, **kwargs)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/lib/python3.4/site-packages/archive_api/viewsets.py" in archive
85. dataset_name = dataset.name.replace(" ", "_")

Exception Type: AttributeError at /api/v1/datasets/27/archive/
Exception Value: 'NoneType' object has no attribute 'replace'
Request information:
USER: valerie

GET: No GET data

POST: No POST data

FILES: No FILES data

COOKIES:
sessionid = 'hvws2k4s5ec0xuhcigjqson43kqz6plj'
_ga = 'GA1.2.925868728.1480364893'
csrftoken = 'ZZ4xcjSMrSJjjISbz590VPuX0e9AOnkUziAbPA9TYzyIeWfmcoJNao9LXuWEaDqV'
tabstyle = 'raw-tab'

META:
CSRF_COOKIE = 'ZZ4xcjSMrSJjjISbz590VPuX0e9AOnkUziAbPA9TYzyIeWfmcoJNao9LXuWEaDqV'
HTTP_ACCEPT = 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,/;q=0.8'
HTTP_ACCEPT_ENCODING = 'gzip, deflate, sdch, br'
HTTP_ACCEPT_LANGUAGE = 'en-US,en;q=0.8'
HTTP_CONNECTION = 'Keep-Alive'
HTTP_COOKIE = '_ga=GA1.2.925868728.1480364893; tabstyle=raw-tab; sessionid=hvws2k4s5ec0xuhcigjqson43kqz6plj; csrftoken=ZZ4xcjSMrSJjjISbz590VPuX0e9AOnkUziAbPA9TYzyIeWfmcoJNao9LXuWEaDqV'
HTTP_HOST = 'ngt-dev.lbl.gov'
HTTP_REFERER = 'https://ngt-dev.lbl.gov/api/v1/datasets/27/'
HTTP_UPGRADE_INSECURE_REQUESTS = '1'
HTTP_USER_AGENT = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.98 Safari/537.36'
HTTP_X_FORWARDED_FOR = '76.220.74.138'
HTTP_X_FORWARDED_HOST = 'ngt-dev.lbl.gov'
HTTP_X_FORWARDED_PROTO = 'https'
HTTP_X_FORWARDED_SERVER = 'ngt-dev.lbl.gov'
PATH_INFO = '/api/v1/datasets/27/archive/'
QUERY_STRING = ''
REMOTE_ADDR = '127.0.0.1'
REQUEST_METHOD = 'GET'
REQUEST_URI = '/api/v1/datasets/27/archive/'
SCRIPT_NAME = ''
SERVER_NAME = 'babe.lbl.gov'
SERVER_PORT = '62000'
SERVER_PROTOCOL = 'HTTP/1.1'
uwsgi.node = b'babe.lbl.gov'
uwsgi.version = b'1.9.17.1-debian'
wsgi.errors = <_io.TextIOWrapper name=2 mode='w' encoding='ANSI_X3.4-1968'>
wsgi.file_wrapper = ''
wsgi.input = <uwsgi._Input object at 0x7fadc7e29600>
wsgi.multiprocess = False
wsgi.multithread = False
wsgi.run_once = False
wsgi.url_scheme = 'http'
wsgi.version =

Settings:
Using settings module ngt_archive.settings
ABSOLUTE_URL_OVERRIDES = {}
ADMINS =
ALLOWED_HOSTS = ['*']
APPEND_SLASH = True
AUTHENTICATION_BACKENDS =
AUTH_LDAP_BIND_DN = 'CN=NGT-Archive,CN=Users,DC=flux,DC=local'
AUTH_LDAP_BIND_PASSWORD = ''
AUTH_LDAP_CACHE_GROUPS = True
AUTH_LDAP_CONNECTION_OPTIONS = {8: 0}
AUTH_LDAP_GROUP_CACHE_TIMEOUT = 300
AUTH_LDAP_SERVER_URI = 'ldap://scooby.lbl.gov'
AUTH_LDAP_USER_ATTR_MAP = {'last_name': 'sn', 'first_name': 'givenName', 'email': 'mail'}
AUTH_LDAP_USER_SEARCH = <django_auth_ldap.config.LDAPSearch object at 0x7fadcb2557b8>
AUTH_PASSWORD_VALIDATORS = ''
AUTH_USER_MODEL = 'auth.User'
BASE_DIR = '/var/www/django/ngt-archive/ngt-dev.lbl.gov/lib/python3.4/site-packages'
CACHES = {'default': {'BACKEND': 'django.core.cache.backends.locmem.LocMemCache'}}
CACHE_MIDDLEWARE_ALIAS = 'default'
CACHE_MIDDLEWARE_KEY_PREFIX = ''
CACHE_MIDDLEWARE_SECONDS = 600
CSRF_COOKIE_AGE = 31449600
CSRF_COOKIE_DOMAIN = None
CSRF_COOKIE_HTTPONLY = False
CSRF_COOKIE_NAME = 'csrftoken'
CSRF_COOKIE_PATH = '/'
CSRF_COOKIE_SECURE = False
CSRF_FAILURE_VIEW = 'django.views.csrf.csrf_failure'
CSRF_HEADER_NAME = 'HTTP_X_CSRFTOKEN'
CSRF_TRUSTED_ORIGINS = []
DATABASES = {'default': {'PASSWORD': '', 'NAME': 'ngt_dev', 'PORT': '', 'AUTOCOMMIT': True, 'OPTIONS': {}, 'TEST': {'COLLATION': None, 'NAME': None, 'CHARSET': None, 'MIRROR': None}, 'ENGINE': 'django.db.backends.postgresql_psycopg2', 'ATOMIC_REQUESTS': False, 'HOST': 'localhost', 'CONN_MAX_AGE': 0, 'TIME_ZONE': None, 'USER': 'ngeetuser'}}
DATABASE_ROUTERS = []
DATASET_ARCHIVE_ROOT = '/data/NGEET/archives/ngt-dev.lbl.gov'
DATASET_ARCHIVE_URL = '/archives/'
DATA_UPLOAD_MAX_MEMORY_SIZE = 2621440
DATA_UPLOAD_MAX_NUMBER_FIELDS = 1000
DATETIME_FORMAT = 'N j, Y, P'
DATETIME_INPUT_FORMATS = ['%Y-%m-%d %H:%M:%S', '%Y-%m-%d %H:%M:%S.%f', '%Y-%m-%d %H:%M', '%Y-%m-%d', '%m/%d/%Y %H:%M:%S', '%m/%d/%Y %H:%M:%S.%f', '%m/%d/%Y %H:%M', '%m/%d/%Y', '%m/%d/%y %H:%M:%S', '%m/%d/%y %H:%M:%S.%f', '%m/%d/%y %H:%M', '%m/%d/%y']
DATE_FORMAT = 'N j, Y'
DATE_INPUT_FORMATS = ['%Y-%m-%d', '%m/%d/%Y', '%m/%d/%y', '%b %d %Y', '%b %d, %Y', '%d %b %Y', '%d %b, %Y', '%B %d %Y', '%B %d, %Y', '%d %B %Y', '%d %B, %Y']
DEBUG = False
DEBUG_PROPAGATE_EXCEPTIONS = False
DECIMAL_SEPARATOR = '.'
DEFAULT_CHARSET = 'utf-8'
DEFAULT_CONTENT_TYPE = 'text/html'
DEFAULT_EXCEPTION_REPORTER_FILTER = 'django.views.debug.SafeExceptionReporterFilter'
DEFAULT_FILE_STORAGE = 'django.core.files.storage.FileSystemStorage'
DEFAULT_FROM_EMAIL = 'webmaster@localhost'
DEFAULT_INDEX_TABLESPACE = ''
DEFAULT_TABLESPACE = ''
DISALLOWED_USER_AGENTS = []
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = 'localhost'
EMAIL_HOST_PASSWORD = ''
EMAIL_HOST_USER = ''
EMAIL_PORT = 25
EMAIL_SSL_CERTFILE = None
EMAIL_SSL_KEYFILE = ''
EMAIL_SUBJECT_PREFIX = '[Django] '
EMAIL_TIMEOUT = None
EMAIL_USE_SSL = False
EMAIL_USE_TLS = False
FILE_CHARSET = 'utf-8'
FILE_UPLOAD_DIRECTORY_PERMISSIONS = None
FILE_UPLOAD_HANDLERS = ['django.core.files.uploadhandler.MemoryFileUploadHandler', 'django.core.files.uploadhandler.TemporaryFileUploadHandler']
FILE_UPLOAD_MAX_MEMORY_SIZE = 2621440
FILE_UPLOAD_PERMISSIONS = None
FILE_UPLOAD_TEMP_DIR = None
FIRST_DAY_OF_WEEK = 0
FIXTURE_DIRS = []
FORCE_SCRIPT_NAME = None
FORMAT_MODULE_PATH = None
IGNORABLE_404_URLS = []
INSTALLED_APPS = ['ngt_archive', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'archive_api.apps.ArchiveApiConfig', 'ui.apps.UiConfig', 'rest_framework']
INTERNAL_IPS = []
LANGUAGES = [('af', 'Afrikaans'), ('ar', 'Arabic'), ('ast', 'Asturian'), ('az', 'Azerbaijani'), ('bg', 'Bulgarian'), ('be', 'Belarusian'), ('bn', 'Bengali'), ('br', 'Breton'), ('bs', 'Bosnian'), ('ca', 'Catalan'), ('cs', 'Czech'), ('cy', 'Welsh'), ('da', 'Danish'), ('de', 'German'), ('dsb', 'Lower Sorbian'), ('el', 'Greek'), ('en', 'English'), ('en-au', 'Australian English'), ('en-gb', 'British English'), ('eo', 'Esperanto'), ('es', 'Spanish'), ('es-ar', 'Argentinian Spanish'), ('es-co', 'Colombian Spanish'), ('es-mx', 'Mexican Spanish'), ('es-ni', 'Nicaraguan Spanish'), ('es-ve', 'Venezuelan Spanish'), ('et', 'Estonian'), ('eu', 'Basque'), ('fa', 'Persian'), ('fi', 'Finnish'), ('fr', 'French'), ('fy', 'Frisian'), ('ga', 'Irish'), ('gd', 'Scottish Gaelic'), ('gl', 'Galician'), ('he', 'Hebrew'), ('hi', 'Hindi'), ('hr', 'Croatian'), ('hsb', 'Upper Sorbian'), ('hu', 'Hungarian'), ('ia', 'Interlingua'), ('id', 'Indonesian'), ('io', 'Ido'), ('is', 'Icelandic'), ('it', 'Italian'), ('ja', 'Japanese'), ('ka', 'Georgian'), ('kk', 'Kazakh'), ('km', 'Khmer'), ('kn', 'Kannada'), ('ko', 'Korean'), ('lb', 'Luxembourgish'), ('lt', 'Lithuanian'), ('lv', 'Latvian'), ('mk', 'Macedonian'), ('ml', 'Malayalam'), ('mn', 'Mongolian'), ('mr', 'Marathi'), ('my', 'Burmese'), ('nb', 'Norwegian Bokmål'), ('ne', 'Nepali'), ('nl', 'Dutch'), ('nn', 'Norwegian Nynorsk'), ('os', 'Ossetic'), ('pa', 'Punjabi'), ('pl', 'Polish'), ('pt', 'Portuguese'), ('pt-br', 'Brazilian Portuguese'), ('ro', 'Romanian'), ('ru', 'Russian'), ('sk', 'Slovak'), ('sl', 'Slovenian'), ('sq', 'Albanian'), ('sr', 'Serbian'), ('sr-latn', 'Serbian Latin'), ('sv', 'Swedish'), ('sw', 'Swahili'), ('ta', 'Tamil'), ('te', 'Telugu'), ('th', 'Thai'), ('tr', 'Turkish'), ('tt', 'Tatar'), ('udm', 'Udmurt'), ('uk', 'Ukrainian'), ('ur', 'Urdu'), ('vi', 'Vietnamese'), ('zh-hans', 'Simplified Chinese'), ('zh-hant', 'Traditional Chinese')]
LANGUAGES_BIDI = ['he', 'ar', 'fa', 'ur']
LANGUAGE_CODE = 'en-us'
LANGUAGE_COOKIE_AGE = None
LANGUAGE_COOKIE_DOMAIN = None
LANGUAGE_COOKIE_NAME = 'django_language'
LANGUAGE_COOKIE_PATH = '/'
LOCALE_PATHS = []
LOGGING = {}
LOGGING_CONFIG = 'logging.config.dictConfig'
LOGIN_REDIRECT_URL = '/accounts/profile/'
LOGIN_URL = '/accounts/login/'
LOGOUT_REDIRECT_URL = None
MANAGERS =
MEDIA_ROOT = ''
MEDIA_URL = ''
MESSAGE_STORAGE = 'django.contrib.messages.storage.fallback.FallbackStorage'
MIDDLEWARE = None
MIDDLEWARE_CLASSES = ['django.middleware.security.SecurityMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.auth.middleware.SessionAuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware']
MIGRATION_MODULES = {}
MONTH_DAY_FORMAT = 'F j'
NUMBER_GROUPING = 0
PASSWORD_HASHERS = ''
PASSWORD_RESET_TIMEOUT_DAYS = ''
PREPEND_WWW = False
REST_FRAMEWORK = {'DEFAULT_PERMISSION_CLASSES': ('rest_framework.permissions.IsAuthenticated', 'rest_framework.permissions.DjangoModelPermissions'), 'DEFAULT_RENDERER_CLASSES': ('rest_framework.renderers.JSONRenderer', 'rest_framework.renderers.BrowsableAPIRenderer'), 'DEFAULT_AUTHENTICATION_CLASSES': ('rest_framework.authentication.SessionAuthentication',), 'DEFAULT_VERSION': 'v1', 'TEST_REQUEST_DEFAULT_FORMAT': 'json', 'DEFAULT_VERSIONING_CLASS': 'rest_framework.versioning.NamespaceVersioning'}
ROOT_URLCONF = 'ngt_archive.urls'
SECRET_KEY = '********************'
SECURE_BROWSER_XSS_FILTER = False
SECURE_CONTENT_TYPE_NOSNIFF = False
SECURE_HSTS_INCLUDE_SUBDOMAINS = False
SECURE_HSTS_SECONDS = 0
SECURE_PROXY_SSL_HEADER =
SECURE_REDIRECT_EXEMPT = []
SECURE_SSL_HOST = None
SECURE_SSL_REDIRECT = False
SERVER_EMAIL = 'root@localhost'
SESSION_CACHE_ALIAS = 'default'
SESSION_COOKIE_AGE = 1209600
SESSION_COOKIE_DOMAIN = None
SESSION_COOKIE_HTTPONLY = True
SESSION_COOKIE_NAME = 'sessionid'
SESSION_COOKIE_PATH = '/'
SESSION_COOKIE_SECURE = False
SESSION_ENGINE = 'django.contrib.sessions.backends.db'
SESSION_EXPIRE_AT_BROWSER_CLOSE = False
SESSION_FILE_PATH = None
SESSION_SAVE_EVERY_REQUEST = False
SESSION_SERIALIZER = 'django.contrib.sessions.serializers.JSONSerializer'
SETTINGS_MODULE = 'ngt_archive.settings'
SHORT_DATETIME_FORMAT = 'm/d/Y P'
SHORT_DATE_FORMAT = 'm/d/Y'
SIGNING_BACKEND = 'django.core.signing.TimestampSigner'
SILENCED_SYSTEM_CHECKS = []
STATICFILES_DIRS =
STATICFILES_FINDERS = ['django.contrib.staticfiles.finders.FileSystemFinder', 'django.contrib.staticfiles.finders.AppDirectoriesFinder']
STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.StaticFilesStorage'
STATIC_ROOT = '/var/www/django/ngt-archive/ngt-dev.lbl.gov/static'
STATIC_URL = '/static/'
TEMPLATES = [{'BACKEND': 'django.template.backends.django.DjangoTemplates', 'APP_DIRS': True, 'OPTIONS': {'context_processors': ['django.template.context_processors.debug', 'django.template.context_processors.request', 'django.contrib.auth.context_processors.auth', 'django.contrib.messages.context_processors.messages']}}]
TEST_NON_SERIALIZED_APPS = []
TEST_RUNNER = 'django.test.runner.DiscoverRunner'
THOUSAND_SEPARATOR = ','
TIME_FORMAT = 'P'
TIME_INPUT_FORMATS = ['%H:%M:%S', '%H:%M:%S.%f', '%H:%M']
TIME_ZONE = 'UTC'
USE_ETAGS = False
USE_I18N = True
USE_L10N = True
USE_THOUSAND_SEPARATOR = False
USE_TZ = True
USE_X_FORWARDED_HOST = False
USE_X_FORWARDED_PORT = False
WSGI_APPLICATION = 'ngt_archive.wsgi.application'
X_FRAME_OPTIONS = 'SAMEORIGIN'
YEAR_MONTH_FORMAT = 'F Y'

Login page for the simple Dataset Prototype milestone.

This error happens after importing data with importcdiac command

Internal Server Error: /api/v1/datasets/

IntegrityError at /api/v1/datasets/
duplicate key value violates unique constraint "archive_api_dataset_pkey"
DETAIL: Key (id)=(3) already exists.

Request Method: POST
Request URL: https://ngt-dev.lbl.gov/api/v1/datasets/
Django Version: 1.10.2
Python Executable: /usr/bin/uwsgi-core
Python Version: 3.4.3
Python Path: ['.', '', '/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/ngt_archive-1.0-py3.4.egg', '/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/djangorestframework_camel_case-0.2.0-py3.4.egg', '/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/django_auth_ldap-1.2.8-py3.4.egg', '/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/pyldap-2.4.25.1-py3.4-linux-x86_64.egg', '/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/django_filter-0.13.0-py3.4.egg', '/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/djangorestframework-3.4.3-py3.4.egg', '/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/Django-1.10.2-py3.4.egg', '/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4', '/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/plat-x86_64-linux-gnu', '/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/lib-dynload', '/usr/lib/python3.4', '/usr/lib/python3.4/plat-x86_64-linux-gnu', '/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages', '/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages']
Server time: Thu, 15 Dec 2016 21:27:40 +0000
Installed Applications:
['ngt_archive',
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'archive_api.apps.ArchiveApiConfig',
'ui.apps.UiConfig',
'rest_framework']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware']

Traceback:

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/Django-1.10.2-py3.4.egg/django/db/backends/utils.py" in execute
64. return self.cursor.execute(sql, params)

  The above exception (duplicate key value violates unique constraint "archive_api_dataset_pkey"

DETAIL: Key (id)=(3) already exists.
) was the direct cause of the following exception:

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/Django-1.10.2-py3.4.egg/django/core/handlers/exception.py" in inner
39. response = get_response(request)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/Django-1.10.2-py3.4.egg/django/core/handlers/base.py" in _legacy_get_response
249. response = self._get_response(request)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/Django-1.10.2-py3.4.egg/django/core/handlers/base.py" in _get_response
187. response = self.process_exception_by_middleware(e, request)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/Django-1.10.2-py3.4.egg/django/core/handlers/base.py" in _get_response
185. response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/Django-1.10.2-py3.4.egg/django/views/decorators/csrf.py" in wrapped_view
58. return view_func(*args, **kwargs)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/djangorestframework-3.4.3-py3.4.egg/rest_framework/viewsets.py" in view
87. return self.dispatch(request, *args, **kwargs)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/djangorestframework-3.4.3-py3.4.egg/rest_framework/views.py" in dispatch
466. response = self.handle_exception(exc)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/djangorestframework-3.4.3-py3.4.egg/rest_framework/views.py" in dispatch
463. response = handler(request, *args, **kwargs)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/djangorestframework-3.4.3-py3.4.egg/rest_framework/mixins.py" in create
21. self.perform_create(serializer)

File "./archive_api/viewsets.py" in perform_create
68. serializer.save(created_by=self.request.user, modified_by=self.request.user)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/djangorestframework-3.4.3-py3.4.egg/rest_framework/serializers.py" in save
191. self.instance = self.create(validated_data)

File "./archive_api/serializers.py" in create
141. dataset = DataSet.objects.create(**validated_data)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/Django-1.10.2-py3.4.egg/django/db/models/manager.py" in manager_method
85. return getattr(self.get_queryset(), name)(*args, **kwargs)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/Django-1.10.2-py3.4.egg/django/db/models/query.py" in create
399. obj.save(force_insert=True, using=self.db)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/Django-1.10.2-py3.4.egg/django/db/models/base.py" in save
796. force_update=force_update, update_fields=update_fields)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/Django-1.10.2-py3.4.egg/django/db/models/base.py" in save_base
824. updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/Django-1.10.2-py3.4.egg/django/db/models/base.py" in _save_table
908. result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/Django-1.10.2-py3.4.egg/django/db/models/base.py" in _do_insert
947. using=using, raw=raw)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/Django-1.10.2-py3.4.egg/django/db/models/manager.py" in manager_method
85. return getattr(self.get_queryset(), name)(*args, **kwargs)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/Django-1.10.2-py3.4.egg/django/db/models/query.py" in _insert
1045. return query.get_compiler(using=using).execute_sql(return_id)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/Django-1.10.2-py3.4.egg/django/db/models/sql/compiler.py" in execute_sql
1054. cursor.execute(sql, params)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/Django-1.10.2-py3.4.egg/django/db/backends/utils.py" in execute
64. return self.cursor.execute(sql, params)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/Django-1.10.2-py3.4.egg/django/db/utils.py" in exit
94. six.reraise(dj_exc_type, dj_exc_value, traceback)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/Django-1.10.2-py3.4.egg/django/utils/six.py" in reraise
685. raise value.with_traceback(tb)

File "/var/www/django/ngt-archive/ngt-dev.lbl.gov/.venv/lib/python3.4/site-packages/Django-1.10.2-py3.4.egg/django/db/backends/utils.py" in execute
64. return self.cursor.execute(sql, params)

Exception Type: IntegrityError at /api/v1/datasets/
Exception Value: duplicate key value violates unique constraint "archive_api_dataset_pkey"
DETAIL: Key (id)=(3) already exists.

Request information:
USER: valerie

GET: No GET data

POST: No POST data

FILES: No FILES data

COOKIES:
_ga = 'GA1.2.925868728.1480364893'
tabstyle = 'raw-tab'
csrftoken = '2kb7olLo8dvBqf5ikKMhdmzrABGQK8ZOVcbb8IuvGr0SgbnI9wETWjwrRxB5rRYT'
sessionid = 'psuq3sw9t9fdwylfmx1vl2idpicssp6j'

META:
CONTENT_LENGTH = '56'
CONTENT_TYPE = 'application/json'
CSRF_COOKIE = '2kb7olLo8dvBqf5ikKMhdmzrABGQK8ZOVcbb8IuvGr0SgbnI9wETWjwrRxB5rRYT'
HTTP_ACCEPT = 'text/html; q=1.0, /'
HTTP_ACCEPT_ENCODING = 'gzip, deflate, br'
HTTP_ACCEPT_LANGUAGE = 'en-US,en;q=0.8'
HTTP_CONNECTION = 'Keep-Alive'
HTTP_COOKIE = '_ga=GA1.2.925868728.1480364893; sessionid=psuq3sw9t9fdwylfmx1vl2idpicssp6j; csrftoken=2kb7olLo8dvBqf5ikKMhdmzrABGQK8ZOVcbb8IuvGr0SgbnI9wETWjwrRxB5rRYT; tabstyle=raw-tab'
HTTP_HOST = 'ngt-dev.lbl.gov'
HTTP_ORIGIN = 'https://ngt-dev.lbl.gov'
HTTP_REFERER = 'https://ngt-dev.lbl.gov/api/v1/datasets/'
HTTP_USER_AGENT = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.98 Safari/537.36'
HTTP_X_CSRFTOKEN = '2kb7olLo8dvBqf5ikKMhdmzrABGQK8ZOVcbb8IuvGr0SgbnI9wETWjwrRxB5rRYT'
HTTP_X_FORWARDED_FOR = '131.243.2.242'
HTTP_X_FORWARDED_HOST = 'ngt-dev.lbl.gov'
HTTP_X_FORWARDED_PROTO = 'https'
HTTP_X_FORWARDED_SERVER = 'ngt-dev.lbl.gov'
HTTP_X_REQUESTED_WITH = 'XMLHttpRequest'
PATH_INFO = '/api/v1/datasets/'
QUERY_STRING = ''
REMOTE_ADDR = '127.0.0.1'
REQUEST_METHOD = 'POST'
REQUEST_URI = '/api/v1/datasets/'
SCRIPT_NAME = ''
SERVER_NAME = 'babe.lbl.gov'
SERVER_PORT = '62000'
SERVER_PROTOCOL = 'HTTP/1.1'
uwsgi.node = b'babe.lbl.gov'
uwsgi.version = b'1.9.17.1-debian'
wsgi.errors = <_io.TextIOWrapper name=2 mode='w' encoding='ANSI_X3.4-1968'>
wsgi.file_wrapper = ''
wsgi.input = <uwsgi._Input object at 0x7f6efdd2df60>
wsgi.multiprocess = False
wsgi.multithread = False
wsgi.run_once = False
wsgi.url_scheme = 'http'
wsgi.version =

Settings:
Using settings module ngt_archive.settings
ABSOLUTE_URL_OVERRIDES = {}
ADMINS =
ALLOWED_HOSTS = ['*']
APPEND_SLASH = True
AUTHENTICATION_BACKENDS =
AUTH_LDAP_BIND_DN = 'CN=NGT-Archive,CN=Users,DC=flux,DC=local'
AUTH_LDAP_BIND_PASSWORD = ''
AUTH_LDAP_CACHE_GROUPS = True
AUTH_LDAP_CONNECTION_OPTIONS = {8: 0}
AUTH_LDAP_GROUP_CACHE_TIMEOUT = 300
AUTH_LDAP_SERVER_URI = 'ldap://scooby.lbl.gov'
AUTH_LDAP_USER_ATTR_MAP = {'last_name': 'sn', 'email': 'mail', 'first_name': 'givenName'}
AUTH_LDAP_USER_SEARCH = <django_auth_ldap.config.LDAPSearch object at 0x7f6f01141ac8>
AUTH_PASSWORD_VALIDATORS = ''
AUTH_USER_MODEL = 'auth.User'
BASE_DIR = '/var/www/django/ngt-archive/ngt-dev.lbl.gov'
CACHES = {'default': {'BACKEND': 'django.core.cache.backends.locmem.LocMemCache'}}
CACHE_MIDDLEWARE_ALIAS = 'default'
CACHE_MIDDLEWARE_KEY_PREFIX = ''
CACHE_MIDDLEWARE_SECONDS = 600
CSRF_COOKIE_AGE = 31449600
CSRF_COOKIE_DOMAIN = None
CSRF_COOKIE_HTTPONLY = False
CSRF_COOKIE_NAME = 'csrftoken'
CSRF_COOKIE_PATH = '/'
CSRF_COOKIE_SECURE = False
CSRF_FAILURE_VIEW = 'django.views.csrf.csrf_failure'
CSRF_HEADER_NAME = 'HTTP_X_CSRFTOKEN'
CSRF_TRUSTED_ORIGINS = []
DATABASES = {'default': {'TEST': {'COLLATION': None, 'NAME': None, 'CHARSET': None, 'MIRROR': None}, 'NAME': 'ngt_dev', 'USER': 'ngeetuser', 'OPTIONS': {}, 'HOST': 'localhost', 'ATOMIC_REQUESTS': False, 'PORT': '', 'TIME_ZONE': None, 'ENGINE': 'django.db.backends.postgresql_psycopg2', 'AUTOCOMMIT': True, 'CONN_MAX_AGE': 0, 'PASSWORD': ''}}
DATABASE_ROUTERS = []
DATASET_ARCHIVE_ROOT = '/data/NGEET/archives/ngt-dev.lbl.gov'
DATASET_ARCHIVE_URL = '/archives/'
DATA_UPLOAD_MAX_MEMORY_SIZE = 2621440
DATA_UPLOAD_MAX_NUMBER_FIELDS = 1000
DATETIME_FORMAT = 'N j, Y, P'
DATETIME_INPUT_FORMATS = ['%Y-%m-%d %H:%M:%S', '%Y-%m-%d %H:%M:%S.%f', '%Y-%m-%d %H:%M', '%Y-%m-%d', '%m/%d/%Y %H:%M:%S', '%m/%d/%Y %H:%M:%S.%f', '%m/%d/%Y %H:%M', '%m/%d/%Y', '%m/%d/%y %H:%M:%S', '%m/%d/%y %H:%M:%S.%f', '%m/%d/%y %H:%M', '%m/%d/%y']
DATE_FORMAT = 'N j, Y'
DATE_INPUT_FORMATS = ['%Y-%m-%d', '%m/%d/%Y', '%m/%d/%y', '%b %d %Y', '%b %d, %Y', '%d %b %Y', '%d %b, %Y', '%B %d %Y', '%B %d, %Y', '%d %B %Y', '%d %B, %Y']
DEBUG = False
DEBUG_PROPAGATE_EXCEPTIONS = False
DECIMAL_SEPARATOR = '.'
DEFAULT_CHARSET = 'utf-8'
DEFAULT_CONTENT_TYPE = 'text/html'
DEFAULT_EXCEPTION_REPORTER_FILTER = 'django.views.debug.SafeExceptionReporterFilter'
DEFAULT_FILE_STORAGE = 'django.core.files.storage.FileSystemStorage'
DEFAULT_FROM_EMAIL = 'webmaster@localhost'
DEFAULT_INDEX_TABLESPACE = ''
DEFAULT_TABLESPACE = ''
DISALLOWED_USER_AGENTS = []
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = 'localhost'
EMAIL_HOST_PASSWORD = ''
EMAIL_HOST_USER = ''
EMAIL_PORT = 25
EMAIL_SSL_CERTFILE = None
EMAIL_SSL_KEYFILE = ''
EMAIL_SUBJECT_PREFIX = '[Django] '
EMAIL_TIMEOUT = None
EMAIL_USE_SSL = False
EMAIL_USE_TLS = False
FILE_CHARSET = 'utf-8'
FILE_UPLOAD_DIRECTORY_PERMISSIONS = None
FILE_UPLOAD_HANDLERS = ['django.core.files.uploadhandler.MemoryFileUploadHandler', 'django.core.files.uploadhandler.TemporaryFileUploadHandler']
FILE_UPLOAD_MAX_MEMORY_SIZE = 2621440
FILE_UPLOAD_PERMISSIONS = None
FILE_UPLOAD_TEMP_DIR = None
FIRST_DAY_OF_WEEK = 0
FIXTURE_DIRS = []
FORCE_SCRIPT_NAME = None
FORMAT_MODULE_PATH = None
IGNORABLE_404_URLS = []
INSTALLED_APPS = ['ngt_archive', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'archive_api.apps.ArchiveApiConfig', 'ui.apps.UiConfig', 'rest_framework']
INTERNAL_IPS = []
LANGUAGES = [('af', 'Afrikaans'), ('ar', 'Arabic'), ('ast', 'Asturian'), ('az', 'Azerbaijani'), ('bg', 'Bulgarian'), ('be', 'Belarusian'), ('bn', 'Bengali'), ('br', 'Breton'), ('bs', 'Bosnian'), ('ca', 'Catalan'), ('cs', 'Czech'), ('cy', 'Welsh'), ('da', 'Danish'), ('de', 'German'), ('dsb', 'Lower Sorbian'), ('el', 'Greek'), ('en', 'English'), ('en-au', 'Australian English'), ('en-gb', 'British English'), ('eo', 'Esperanto'), ('es', 'Spanish'), ('es-ar', 'Argentinian Spanish'), ('es-co', 'Colombian Spanish'), ('es-mx', 'Mexican Spanish'), ('es-ni', 'Nicaraguan Spanish'), ('es-ve', 'Venezuelan Spanish'), ('et', 'Estonian'), ('eu', 'Basque'), ('fa', 'Persian'), ('fi', 'Finnish'), ('fr', 'French'), ('fy', 'Frisian'), ('ga', 'Irish'), ('gd', 'Scottish Gaelic'), ('gl', 'Galician'), ('he', 'Hebrew'), ('hi', 'Hindi'), ('hr', 'Croatian'), ('hsb', 'Upper Sorbian'), ('hu', 'Hungarian'), ('ia', 'Interlingua'), ('id', 'Indonesian'), ('io', 'Ido'), ('is', 'Icelandic'), ('it', 'Italian'), ('ja', 'Japanese'), ('ka', 'Georgian'), ('kk', 'Kazakh'), ('km', 'Khmer'), ('kn', 'Kannada'), ('ko', 'Korean'), ('lb', 'Luxembourgish'), ('lt', 'Lithuanian'), ('lv', 'Latvian'), ('mk', 'Macedonian'), ('ml', 'Malayalam'), ('mn', 'Mongolian'), ('mr', 'Marathi'), ('my', 'Burmese'), ('nb', 'Norwegian Bokmål'), ('ne', 'Nepali'), ('nl', 'Dutch'), ('nn', 'Norwegian Nynorsk'), ('os', 'Ossetic'), ('pa', 'Punjabi'), ('pl', 'Polish'), ('pt', 'Portuguese'), ('pt-br', 'Brazilian Portuguese'), ('ro', 'Romanian'), ('ru', 'Russian'), ('sk', 'Slovak'), ('sl', 'Slovenian'), ('sq', 'Albanian'), ('sr', 'Serbian'), ('sr-latn', 'Serbian Latin'), ('sv', 'Swedish'), ('sw', 'Swahili'), ('ta', 'Tamil'), ('te', 'Telugu'), ('th', 'Thai'), ('tr', 'Turkish'), ('tt', 'Tatar'), ('udm', 'Udmurt'), ('uk', 'Ukrainian'), ('ur', 'Urdu'), ('vi', 'Vietnamese'), ('zh-hans', 'Simplified Chinese'), ('zh-hant', 'Traditional Chinese')]
LANGUAGES_BIDI = ['he', 'ar', 'fa', 'ur']
LANGUAGE_CODE = 'en-us'
LANGUAGE_COOKIE_AGE = None
LANGUAGE_COOKIE_DOMAIN = None
LANGUAGE_COOKIE_NAME = 'django_language'
LANGUAGE_COOKIE_PATH = '/'
LOCALE_PATHS = []
LOGGING = {}
LOGGING_CONFIG = 'logging.config.dictConfig'
LOGIN_REDIRECT_URL = '/accounts/profile/'
LOGIN_URL = '/accounts/login/'
LOGOUT_REDIRECT_URL = None
MANAGERS =
MEDIA_ROOT = ''
MEDIA_URL = ''
MESSAGE_STORAGE = 'django.contrib.messages.storage.fallback.FallbackStorage'
MIDDLEWARE = None
MIDDLEWARE_CLASSES = ['django.middleware.security.SecurityMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.auth.middleware.SessionAuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware']
MIGRATION_MODULES = {}
MONTH_DAY_FORMAT = 'F j'
NUMBER_GROUPING = 0
PASSWORD_HASHERS = ''
PASSWORD_RESET_TIMEOUT_DAYS = ''
PREPEND_WWW = False
REST_FRAMEWORK = {'TEST_REQUEST_DEFAULT_FORMAT': 'json', 'DEFAULT_PERMISSION_CLASSES': ('rest_framework.permissions.IsAuthenticated', 'rest_framework.permissions.DjangoModelPermissions'), 'DEFAULT_VERSIONING_CLASS': 'rest_framework.versioning.NamespaceVersioning', 'DEFAULT_VERSION': 'v1', 'DEFAULT_AUTHENTICATION_CLASSES': ('rest_framework.authentication.SessionAuthentication',), 'DEFAULT_RENDERER_CLASSES': ('rest_framework.renderers.JSONRenderer', 'rest_framework.renderers.BrowsableAPIRenderer')}
ROOT_URLCONF = 'ngt_archive.urls'
SECRET_KEY = '********************'
SECURE_BROWSER_XSS_FILTER = False
SECURE_CONTENT_TYPE_NOSNIFF = False
SECURE_HSTS_INCLUDE_SUBDOMAINS = False
SECURE_HSTS_SECONDS = 0
SECURE_PROXY_SSL_HEADER =
SECURE_REDIRECT_EXEMPT = []
SECURE_SSL_HOST = None
SECURE_SSL_REDIRECT = False
SERVER_EMAIL = 'root@localhost'
SESSION_CACHE_ALIAS = 'default'
SESSION_COOKIE_AGE = 1209600
SESSION_COOKIE_DOMAIN = None
SESSION_COOKIE_HTTPONLY = True
SESSION_COOKIE_NAME = 'sessionid'
SESSION_COOKIE_PATH = '/'
SESSION_COOKIE_SECURE = False
SESSION_ENGINE = 'django.contrib.sessions.backends.db'
SESSION_EXPIRE_AT_BROWSER_CLOSE = False
SESSION_FILE_PATH = None
SESSION_SAVE_EVERY_REQUEST = False
SESSION_SERIALIZER = 'django.contrib.sessions.serializers.JSONSerializer'
SETTINGS_MODULE = 'ngt_archive.settings'
SHORT_DATETIME_FORMAT = 'm/d/Y P'
SHORT_DATE_FORMAT = 'm/d/Y'
SIGNING_BACKEND = 'django.core.signing.TimestampSigner'
SILENCED_SYSTEM_CHECKS = []
STATICFILES_DIRS =
STATICFILES_FINDERS = ['django.contrib.staticfiles.finders.FileSystemFinder', 'django.contrib.staticfiles.finders.AppDirectoriesFinder']
STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.StaticFilesStorage'
STATIC_ROOT = '/var/www/django/ngt-archive/ngt-dev.lbl.gov/static'
STATIC_URL = '/static/'
TEMPLATES = [{'OPTIONS': {'context_processors': ['django.template.context_processors.debug', 'django.template.context_processors.request', 'django.contrib.auth.context_processors.auth', 'django.contrib.messages.context_processors.messages']}, 'BACKEND': 'django.template.backends.django.DjangoTemplates', 'APP_DIRS': True}]
TEST_NON_SERIALIZED_APPS = []
TEST_RUNNER = 'django.test.runner.DiscoverRunner'
THOUSAND_SEPARATOR = ','
TIME_FORMAT = 'P'
TIME_INPUT_FORMATS = ['%H:%M:%S', '%H:%M:%S.%f', '%H:%M']
TIME_ZONE = 'UTC'
USE_ETAGS = False
USE_I18N = True
USE_L10N = True
USE_THOUSAND_SEPARATOR = False
USE_TZ = True
USE_X_FORWARDED_HOST = False
USE_X_FORWARDED_PORT = False
WSGI_APPLICATION = 'ngt_archive.wsgi.application'
X_FRAME_OPTIONS = 'SAMEORIGIN'
YEAR_MONTH_FORMAT = 'F Y'

When creating a dataset with name and description, the name value is not POST to the database.

Time zones are not enabled in the application.

https://docs.djangoproject.com/en/1.10/topics/i18n/timezones/

Support for application deployment is needed.

• versioning from git tags (version.py is generated not maintained in git)
• building a complete distribution ( should package all sources necessary for a complete deployment)
• documentation (general documentaion decribing how version a deployment and to prepare a django deployment directory)

Database integrity errors are returned as INternal server errors (500). They should have a nicer handling.

(.venv)vagrant@ubuntu-14:/vagrant$ tail -100f ngt-archive.log 
    response = self._get_response(request)
  File "/vagrant/.venv/lib/python3.4/site-packages/django/core/handlers/base.py", line 187, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/vagrant/.venv/lib/python3.4/site-packages/django/core/handlers/base.py", line 185, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/vagrant/.venv/lib/python3.4/site-packages/django/views/decorators/csrf.py", line 58, in wrapped_view
    return view_func(*args, **kwargs)
  File "/vagrant/.venv/lib/python3.4/site-packages/rest_framework/viewsets.py", line 83, in view
    return self.dispatch(request, *args, **kwargs)
  File "/vagrant/.venv/lib/python3.4/site-packages/rest_framework/views.py", line 477, in dispatch
    response = self.handle_exception(exc)
  File "/vagrant/.venv/lib/python3.4/site-packages/rest_framework/views.py", line 437, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/vagrant/.venv/lib/python3.4/site-packages/rest_framework/views.py", line 474, in dispatch
    response = handler(request, *args, **kwargs)
  File "/vagrant/.venv/lib/python3.4/site-packages/rest_framework/mixins.py", line 21, in create
    self.perform_create(serializer)
  File "./archive_api/viewsets.py", line 68, in perform_create
    serializer.save(created_by=self.request.user, modified_by=self.request.user)
  File "/vagrant/.venv/lib/python3.4/site-packages/rest_framework/serializers.py", line 214, in save
    self.instance = self.create(validated_data)
  File "./archive_api/serializers.py", line 141, in create
    dataset = DataSet.objects.create(**validated_data)
  File "/vagrant/.venv/lib/python3.4/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/vagrant/.venv/lib/python3.4/site-packages/django/db/models/query.py", line 399, in create
    obj.save(force_insert=True, using=self.db)
  File "/vagrant/.venv/lib/python3.4/site-packages/django/db/models/base.py", line 796, in save
    force_update=force_update, update_fields=update_fields)
  File "/vagrant/.venv/lib/python3.4/site-packages/django/db/models/base.py", line 824, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/vagrant/.venv/lib/python3.4/site-packages/django/db/models/base.py", line 908, in _save_table
    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
  File "/vagrant/.venv/lib/python3.4/site-packages/django/db/models/base.py", line 947, in _do_insert
    using=using, raw=raw)
  File "/vagrant/.venv/lib/python3.4/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/vagrant/.venv/lib/python3.4/site-packages/django/db/models/query.py", line 1045, in _insert
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/vagrant/.venv/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 1054, in execute_sql
    cursor.execute(sql, params)
  File "/vagrant/.venv/lib/python3.4/site-packages/django/db/backends/utils.py", line 79, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/vagrant/.venv/lib/python3.4/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/vagrant/.venv/lib/python3.4/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/vagrant/.venv/lib/python3.4/site-packages/django/utils/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/vagrant/.venv/lib/python3.4/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: duplicate key value violates unique constraint "archive_api_dataset_pkey"
DETAIL:  Key (id)=(6) already exists.

Upload and storage of dataset files. This should support both local upload. Locally from the desktop/laptop.

• Only zip files are allowed
• Only the latest uploaded file can be downloaded. Previously downloaded files will be accessible from the file system
• Data files are guarded by Dataset access levels (private, NGEE Tropics, Public)
• User will be allowed to download their uploaded file from a login protected url (see snippet)
  { "url": "https://localhost:9443/api/v1/datasets/23/", "data_set_id": "NGT23", "name": "PR-Sabana Meteorological Data 01-2015 to 09-2016", ... "archive": "https://localhost:9443/api/v1/datasets/23/archive/", "created_by": "admin", "created_date": "2016-12-06T18:06:57.053215Z", "modified_by": "admin", "modified_date": "2016-12-06T20:33:06.986067Z", "cdiac_import": true, "cdiac_submission_contact": "https://localhost:9443/api/v1/people/42/" }
• When archive link above is accessed - a file named NGT23_1.0_PRSabana_Meteorological_Data_012015_to_092016.zip will be downloaded. The file name pattern is <dataset_id><dataset_version><dataset_name>.<file_extension>.
• data_set_id format the number with leading zeros for better sorting and readabilit

Manage institutions in a separate table to avoid inconsistent institutions being entered by users (e.g. LBNL, LBL, Lawrence Berkeley National Lab, Berkeley Lab etc.)

Also, update dataset edit page to create authors and contacts with institutions

Update Dataset metadata model to support future versioning of Datasets. This will involve setting default version to 0 for a new DRAFT. Also, dataset_id and version together should be unique which will require dataset id not being a derived field (from internal Dataset.id)

This requires the underlying model of the dataset to renamed from 'owner' to 'creator'. The json needs to return 'createdBy' and 'createdByDate'

The virtual machine development environment should have an example of using a maintenance page.

REST API and admin management pages for data model entities that support dataset archival. Models include: Sites, Plots, Contacts, MeasurementVariables.

REST API

• /sites - GET the list of sites
• /sites/:site_id - GET a single site
• /plots - GET all the plots
• /plots/:plot_id - GET a single plot
• /contacts: GET all contacts, POST a contact
• /contacts/:id - GET,PUT a single contact
• /variables - GET all measurement variables
• /variables/:id - GET a measurement variable

Default User groups: NGT Administrator, NGT User

This is autogenerated after submission. Drafts shouldn't have a published dataset ID.. (there could be a hidden field in the database that users never see).

Related to #15

Setup an apache configuration that allows apache to host the static doi pages.

Should the name and description (possibly a few other fields) be required while creating a data set?

Submission data is only a date field

User Login (Active Directory) using LDAP Authentication

The dataSetId needs to be returned for a dataset (NGT#)

Incorrectly resolving qaqc status and access level in importcdiac. Access level is not being set. QAQC status is not processing all the status choices

Remove djangorestframework-camel-case. This has several bugs which need to be fixed. The project seems to be dead so we are going to drop it.

The version field is missing from the model. This will be hard coded to 1.0 for the initial release

We need to get feedback on what the best format for the view of public datasets is

The archive service needs SSL support in order to securly handle authentication. The configuration needs to deployment to apache with SSL

Currently the admin and base site have login pages that reference Django admin and Django rest framework. These should reference NGEE Tropics Archive

Proof of concept with a simple example that exercises auth, rest backend. The prototype will be a very simple DataSet model (name, description), a REST API for (GET,PUT,POST) creating, editing and listing datasets.

Upload filetypes needs to be expanded to other archive types

The uploaded dataset files should reside in a folder structure. This will make it more manageable for backing up to HPSS and navigating.

• dataset_id
  • version
    • archive file(s) - the archive file could have been uploaded more than once. The old copies are kept around.

Basic Data UI functionality with creation, upload, viewing of datasets, sites and plots. Editing capabilities not complete but and example of a modal editing page

Read the metadata supplied by the REST API. This will inform the UI about the datatypes, constraints and other properties required to select the correct UI widget.

Proof of concept with a simple example that exercises auth, javascript client, rest backend. The prototype will be a very simple DataSet model (name, description), a REST API for (GET,PUT,POST) creating, editing and listing datasets with a Javascript client as the front end.

Basic Django Application setup with configuration etc

We have had an occassional case or two where users want to change things after it's been approved. I know this creates problems since we need the versioning, but the way we got around that for the first round was to allow it to be returned to submission mode and then have the admin make the changes.

Related to #15

The submission contact is not necessarily the creator of the dataset. This means that a new field called sumission contact should be added to the field.

Environment:

Request Method: GET
Request URL: http://localhost:9999/api/v1/datasets/

Django Version: 1.10.3
Python Version: 3.4.3
Installed Applications:
['ngt_archive',
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'archive_api.apps.ArchiveApiConfig',
'rest_framework']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware']

Traceback:

File "/vagrant/.venv/lib/python3.4/site-packages/django/core/handlers/exception.py" in inner
39. response = get_response(request)

File "/vagrant/.venv/lib/python3.4/site-packages/django/core/handlers/base.py" in _legacy_get_response
249. response = self._get_response(request)

File "/vagrant/.venv/lib/python3.4/site-packages/django/core/handlers/base.py" in _get_response
187. response = self.process_exception_by_middleware(e, request)

File "/vagrant/.venv/lib/python3.4/site-packages/django/core/handlers/base.py" in _get_response
185. response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/vagrant/.venv/lib/python3.4/site-packages/django/views/decorators/csrf.py" in wrapped_view
58. return view_func(*args, **kwargs)

File "/vagrant/.venv/lib/python3.4/site-packages/rest_framework/viewsets.py" in view
83. return self.dispatch(request, *args, **kwargs)

File "/vagrant/.venv/lib/python3.4/site-packages/rest_framework/views.py" in dispatch
477. response = self.handle_exception(exc)

File "/vagrant/.venv/lib/python3.4/site-packages/rest_framework/views.py" in handle_exception
437. self.raise_uncaught_exception(exc)

File "/vagrant/.venv/lib/python3.4/site-packages/rest_framework/views.py" in dispatch
474. response = handler(request, *args, **kwargs)

File "/vagrant/.venv/lib/python3.4/site-packages/rest_framework/mixins.py" in list
48. return Response(serializer.data)

File "/vagrant/.venv/lib/python3.4/site-packages/rest_framework/serializers.py" in data
725. ret = super(ListSerializer, self).data

File "/vagrant/.venv/lib/python3.4/site-packages/rest_framework/serializers.py" in data
262. self._data = self.to_representation(self.instance)

File "/vagrant/.venv/lib/python3.4/site-packages/rest_framework/serializers.py" in to_representation
643. self.child.to_representation(item) for item in iterable

File "/vagrant/.venv/lib/python3.4/site-packages/rest_framework/serializers.py" in
643. self.child.to_representation(item) for item in iterable

File "/vagrant/.venv/lib/python3.4/site-packages/rest_framework/serializers.py" in to_representation
479. fields = self._readable_fields

File "/vagrant/.venv/lib/python3.4/site-packages/django/utils/functional.py" in get
35. res = instance.dict[self.name] = self.func(instance)

File "/vagrant/.venv/lib/python3.4/site-packages/rest_framework/serializers.py" in _readable_fields
373. field for field in self.fields.values()

File "/vagrant/.venv/lib/python3.4/site-packages/rest_framework/serializers.py" in fields
359. for key, value in self.get_fields().items():

File "/vagrant/.venv/lib/python3.4/site-packages/rest_framework/serializers.py" in get_fields
980. field_names = self.get_field_names(declared_fields, info)

File "/vagrant/.venv/lib/python3.4/site-packages/rest_framework/serializers.py" in get_field_names
1053. serializer_class=self.class.name

Exception Type: AssertionError at /api/v1/datasets/
Exception Value: ("Creating a ModelSerializer without either the 'fields' attribute or the 'exclude' attribute has been deprecated since 3.3.0, and is now disallowed. Add an explicit fields = 'all' to the DataSetSerializer serializer.",)

The order of the dataset authors is important. The order in which the authors are specified must be preserved.

File "/private/tmp/ngt-archive/archive_api/viewsets.py", line 156, in <module> class DataSetViewSet(ModelViewSet): File "/private/tmp/ngt-archive/archive_api/viewsets.py", line 165, in DataSetViewSet parser_classes = (CamelCaseJSONParser, MultiPartParser, FormParser) NameError: name 'CamelCaseJSONParser' is not defined

Currently I can create datasets with both dates set in the future. Wanted to get this clarified.

Is it possible to add accepted file_types to http OPTIONS of the Dataset.archive?

This issue relates to the output of the OPTIONS call to api/v1/datasets/. These are some of the proposed enhancements/additional parameters:

"multiple": boolean
Allows for multiple values to be added

"format": "yy-mm-dd", "ISO"
Used by Javascript/Jquery to understand the formatting of date and time variables

"searchable_field": boolean
Used to create a searchable list which supports typeahead

"add_new": boolean
If a searchable list doesn't contain the search term, a new value will be created automatically. This field is always used in conjunction with "searcahable_field"

For parameters such as sites and contacts, suggest changing type to "reference_list". This will allow me to create a fetch the list of sites and contacts and create a dropdown menu. This field can be used in conjunction with "searchable_field".

Admins should also be able to submit Datasets

Related to #15

NGT admins should be allowed to edit in draft mode.

Related to #15

The Dataset entity is missing the originating institution field.

Dataset Originating Institution: Name of the organization that performed the research or issued the dataset. Separate multiple organizations with a semicolon and a space.