akvo / akvo-flow Goto Github PK

Because this tool is too complex for most users to execute, can we remove this from the Dashboard UI?

This would be in at least two places:

1. Admin Home > Import Data > Import Survey
2. Survey Manager > Question List > Import Questions > Remove "File" option from Source dropdown.

We will remove this functionality for 1.5, but would like to do it for the Dashboard 1.0 that's used in the short term FLOW pilots if possible.

To make sure instances have all the latest features and fixes for easier instance management.

Active on device, need to add SCAN to the list of question types in the edit question screen on the Dashboard UI.

As a FLOW Partner Support team staff member, I need to be able to create and edit FLOW maps and pop up windows from the Dashboard without developer intervention.

New legend design (UI element)

Build new utility that can take the output from the zips and generate the raw data spreadsheet without going through the server at all.

Wrap it up in an applet and make it reusable in new hidden Admin Tools/Admin Only menu.

How to reproduce the problem:

1. in http://ircflow.appspot.com/Dashboard.html, go to to survey group 'BKF', survey 'AUE v.2 april 1'.
2. click on the first question group 'Identification'.
3. click on 'Next 2: Organisation de l'AUE>'
4. click on Next 5 times, this leads to 'previous: 5: Les Pannes' and 'next: 5:Gestion au sein de l'AEU'. This is the wrong result

Expected behaviour:
The navigation should cycle through all 9 items.

How to reproduce
in a survey on a phone, fill in a barcode text field directly. If the question is marked mandatory, a 'missing mandatory question' error will be raised when you try to submit, even when there is content in the field.

Expected behaviour
The question should be regarded as completed, based on the presence of text in the field.

Build automated instance creation and update tools to be ready when we reach 20-30 instances for FLOW 1.5.

To meet our commitment to making the FLOW codebase open source by end of Q2 we have the following remaining tasks:

• ensure open source license text is included in all source file headers
• create example/template secrets files instead of storing the actual files in our Github repo

The burkina team are seeing replacement characters in french text in the xlsx export, see attached file. Could this be due to a UTF-8 encoding problem in the xlsx export module? The strange characters also show up in the backend.

Instance: IRC
survey group: BKF
survey: Usager au point d'eau v.2 April 1

The raw data report generation in the IRC instance fails on the survey above. Other surveys in the same group generate the report without problems.

The behavior was confirmed by me. I tried three times, and the same error message is generated during the "Loading Instance details" phase: "java.io.IOException: Server returned HTTP response code: 500 for URL: http://ircflow.appspot.com/databackout?action=listInstance&surveyId=942033&includeDate=true"

------------------ last part trace ---------------
Calling: http://ircflow.appspot.com/surveyrestapi?action=listQuestions&questionGroupId=942038
response: {"message":null,"cursor":null,"resultCount":0,"class":"class org.waterforpeople.mapping.app.web.dto.SurveyRestResponse","code":null,"offset":0,"url":null,"dtoList":[{"translationMap":null,"allowDecimal":false,"optionContainerDto":null,"type":{"declaringClass":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType","class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType"},"tip":"","maxVal":0,"order":1,"isName":false,"questionHelpList":null,"collapseable":false,"path":"BKF/Usager au point d'eau v.2 april 1/Observations","allowMultipleFlag":false,"immutable":false,"allowOtherFlag":false,"allowSign":false,"text":"11. Observations","minVal":0,"class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto","questionDependency":null,"mandatoryFlag":false,"questionGroupId":942038,"questionTypeString":"FREE_TEXT","surveyId":942033,"displayName":"11. Observations","keyId":941034}]}

Calling: http://ircflow.appspot.com/databackout?action=listInstance&surveyId=942033&includeDate=true
java.io.IOException: Server returned HTTP response code: 500 for URL: http://ircflow.appspot.com/databackout?action=listInstance&surveyId=942033&includeDate=true
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1436)
at org.waterforpeople.mapping.dataexport.service.BulkDataServiceClient.fetchDataFromServerGET(BulkDataServiceClient.java:1073)
at org.waterforpeople.mapping.dataexport.service.BulkDataServiceClient.fetchDataFromServer(BulkDataServiceClient.java:986)
at org.waterforpeople.mapping.dataexport.service.BulkDataServiceClient.fetchInstanceIds(BulkDataServiceClient.java:283)
at org.waterforpeople.mapping.dataexport.GraphicalSurveySummaryExporter.fetchAndWriteRawData(GraphicalSurveySummaryExporter.java:348)
at org.waterforpeople.mapping.dataexport.GraphicalSurveySummaryExporter.export(GraphicalSurveySummaryExporter.java:268)
at com.gallatinsystems.framework.dataexport.applet.DataExportAppletImpl.doExport(DataExportAppletImpl.java:57)
at com.gallatinsystems.framework.dataexport.applet.DataExportAppletImpl.init(DataExportAppletImpl.java:31)
at sun.plugin2.applet.Plugin2Manager$AppletExecutionRunnable.run(Plugin2Manager.java:1639)
at java.lang.Thread.run(Thread.java:680)

As a FLOW partner (WFP), I need to be able to conduct surveys on the device in the additional languages of Chichewa, Bengali, Hindi, and Quechua.

In the Akvo Instance, in survey group RWHBSPNEPAL, in the survey Sindupalchowk 2012 (511008)

When following the mapping data workflow, we followed these steps:

1. we created a metric called "satisfaction" in the metric manager
2. we went to the survey question "5. Satisfaction on RWH system", in "User perception" question group and assigned the metric 'Satisfaction' to the question and saved the survey.
3. we published the survey.

Unexpected behaviour: When going back to the question (after refresh of dashboard), the metric has not saved.

One observation is that in the View Messages window, the survey we have edited shows up under its old name, "12345/SALYAN2012".

We checked the XML and confirmed it is the same survey.

When trying to run the Comprehensive Report on the Household survey for in the Afram Plains survey group from the Crepa instance, the reports is running to completion, but not publishing / saving the results to disk.

The issue was reported by Keri, and confirmed by Mark.

When investigating in the backend, it is clear a lot of resources are used (4 instances, 400MB of memory).

Keri included her Java console messages, which show a Java memory exception.

---------------- report by Keri ---------------
I am working in the CREPA instance. Browser is Firefox. I am trying to run the Comprehensive Report on the Household survey for in the Afram Plains survey group. I was able to run the same report for the Waterpoint and Watsan surveys without an issue but when I try to run household, it tells me the report is completed but it never appears in the folder where I had saved it and when I run the java console along with the report I get this exception:

Exception in thread "thread applet-com.gallatinsystems.framework.dataexport.applet.DataExportAppletImpl-2" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Unknown Source)
at java.io.ByteArrayOutputStream.write(Unknown Source)
at javax.imageio.stream.FileCacheImageOutputStream.flushBefore(Unknown Source)
at com.sun.imageio.plugins.png.IDATOutputStream.finishChunk(Unknown Source)
at com.sun.imageio.plugins.png.IDATOutputStream.deflate(Unknown Source)
at com.sun.imageio.plugins.png.IDATOutputStream.finish(Unknown Source)
at com.sun.imageio.plugins.png.PNGImageWriter.write_IDAT(Unknown Source)
at com.sun.imageio.plugins.png.PNGImageWriter.write(Unknown Source)
at javax.imageio.ImageWriter.write(Unknown Source)
at javax.imageio.ImageIO.write(Unknown Source)
at javax.imageio.ImageIO.write(Unknown Source)
at org.jfree.chart.encoders.SunPNGEncoderAdapter.encode(SunPNGEncoderAdapter.java:127)
at org.jfree.chart.encoders.SunPNGEncoderAdapter.encode(SunPNGEncoderAdapter.java:108)
at org.jfree.chart.encoders.EncoderUtil.encode(EncoderUtil.java:66)
at org.jfree.chart.ChartUtilities.encodeAsPNG(ChartUtilities.java:642)
at com.gallatinsystems.common.util.JFreechartChartUtil.getPieChart(JFreechartChartUtil.java:42)
at org.waterforpeople.mapping.dataexport.GraphicalSurveySummaryExporter.writeSummaryReport(GraphicalSurveySummaryExporter.java:754)
at org.waterforpeople.mapping.dataexport.GraphicalSurveySummaryExporter.export(GraphicalSurveySummaryExporter.java:294)
at com.gallatinsystems.framework.dataexport.applet.DataExportAppletImpl.doExport(DataExportAppletImpl.java:57)
at com.gallatinsystems.framework.dataexport.applet.DataExportAppletImpl.init(DataExportAppletImpl.java:31)
at sun.plugin2.applet.Plugin2Manager$AppletExecutionRunnable.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

Trying to generate a bootstrap file for Nepal BSP/Roof Top Rainwater Harvesting in Houses survey. Complete Dashboard form but email to user with survey file attached never arrives.

Survey is published and will assign to phones (with connectivity, tried by Caetie), but field users not able to use this function and need to be able to bootstrap.

As a FLOW Dashboard user, I want to be able to sort my Raw Data Reports by latitude, longitude, elevation and/or unique code, but this needs to be backwards compatible with previous report versions where the columns are still compound for import and export.

Requirements:

• Break up response column for Geo question types into 4 columns for latitude, longitude, elevation, and unique code so that they can be sortable.
• Ensure backward compatibility so that older reports with the compound column can still import and export with newer reports that have the columns broken up.

Need either a method or a tool to transfer surveys and data between FLOW instances while we get our partners set up on the Akvo instance -- and then will need this again when we transfer them to their own instances.

Instance: watermapmonitordev (WFP instance)
survey group: 2012 surveys

The geographical roll ups are not being generated in the comprehensive report on the Survey 2012 / Uganda Water Point Survey 2012.

Other comprehensive reports (India Water points 2012, Honduras 2011 Water points) do contain the roll ups.

The behavior was confirmed by both Keri and Mark.

The logs seem to show similar structure for both the Uganda and the India report creation.

The java console doesn't show any conspicuous behaviour.

The logs do show one thing that seems strange, but this is shown during both the Uganda and the India report generation:

"2012-05-23 03:24:24.142 /remoteexception 200 334ms 0kb Apache-HttpClient/UNAVAILABLE (java 1.4)
112.79.36.91 - - [23/May/2012:03:24:24 -0700] "POST /remoteexception HTTP/1.1" 200 41 - "Apache-HttpClient/UNAVAILABLE (java 1.4)" "watermapmonitordev.appspot.com" ms=334 cpu_ms=350 api_cpu_ms=257 cpm_usd=0.009881 instance=00c61b117c71be4f464e7be04e95712dea26b8"

-------------------- Original report by Keri ----------------

Andrew flagged a problem with the WFP instance on the comprehensive reports. I confirmed the problem. I am using Firefox as the browser. I am having an odd problem when I pull the comp reports for the 2012 survey group, Uganda surveys they just have the raw data and the summary page, no geographical roll ups. I pulled them several times to check. This happens with all three Uganda surveys (Waterpoint, Household and PI) I also pulled other reports (india 2012 waterpoint and Honduras 2011 Water Point) and they pull correctly with the roll ups. When running the Uganda reports I got no error messages and the report seemed to go through all 7 steps without a problem. I checked the java console and didn't see any obvious errors. I attaching both the problem Uganda reports and the India one that ran fine.

As a FLOW user, I need to be able to control the flow of survey data onto my public facing maps and make choices about what data appear in my map's pop-up windows in a user interface on my FLOW Dashboard [Self-service maps phase 2]

When trying to re- import an xls or xlsx of cleaned test data for Test Surveys/CMO Test Survey on the WFP instance, the data appear to not be importing correctly, with only the geo question appearing in Raw Data Manager after the import for the cleaned data records.

Three files attached for illustration:

1. Initial xls raw data report
2. cleaned xls data report
3. cleaned xlsx data report

Java version 1.6.0_31 on Mac OS 10.6.8

As a FLOW user, I want to be able to run a report containing detailed data analysis that doesn't take a large amount of processing and with a manageable file size.

Possible implementation: make pie charts optional in pop-up window, similar to how geographic roll-ups are now.

Integrate warning message about processing time and memory if user tries to run report with pie charts.

Chris,

could Nepal be added in the right places, like was done for Uganda? It is urgent - Amitangshu is in Nepal this week.

thanks
mark

From both field and hq, the Water For People instance is not loading properly. When log on to page, see only Dashboard portlets, no header image or menu bar.

Multiple browsers tried: Firefox, Chrome

a second test issue. this one is a bug.

National (= level 0)
Regional (= level 1)
District (= level 2)

Need to locate + load GIS shapeflles for all program countries, then display on maps as part of default view.

This is needed in reporting as well, can currently be achieved through including admin areas in surveys and rolling up by geoname.

Need new template for map pop-up windows.

Device IDs (set on each device in Preferences) are not currently showing up consistently on Dashboard.

Currently, the device sends its Device ID when it submits a survey, but a server code overwrite may be happening. Need to investigate paths that are touching that data to see where it's being possibly overwritten.

Keri at WFP tried to run a Google Earth report from her instance that ran for a while but would not complete, the resulting kmz file is blank.

She's running Java 1.6.0_31

I checked their dynamic instances on the AppEngine Dashboard and they do not have a large number of errors.

I tried from my machine (with the same Java version) and got the same error.

Java Console readout below (only the error portion at the end):

Fetching next set of records for: BO : 21
Calling: http://watermapmonitordev.appspot.com/placemarkrestapi?needDetailsF...
Starting to process: BO
Fetching next set of records for: BO : 22
Calling: http://watermapmonitordev.appspot.com/placemarkrestapi?needDetailsF...
Starting to process: BO
Fetching next set of records for: BO : 23
Calling: http://watermapmonitordev.appspot.com/placemarkrestapi?needDetailsF...
Starting to process: BO
Fetching next set of records for: BO : 24
Calling: http://watermapmonitordev.appspot.com/placemarkrestapi?needDetailsF...
Starting to process: BO
Fetching next set of records for: BO : 25
Calling: http://watermapmonitordev.appspot.com/placemarkrestapi?needDetailsF...
Starting to process: BO
Fetching next set of records for: BO : 26
Calling: http://watermapmonitordev.appspot.com/placemarkrestapi?needDetailsF...
java.io.IOException: Server returned HTTP response code: 500 for URL: http://watermapmonitordev.appspot.com/placemarkrestapi?needDetailsF... Server returned HTTP response code: 500 for URL: http://watermapmonitordev.appspot.com/placemarkrestapi?needDetailsF...
java.io.IOException: Server returned HTTP response code: 500 for URL: http://watermapmonitordev.appspot.com/placemarkrestapi?needDetailsF...

at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at org.waterforpeople.mapping.dataexport.service.BulkDataServiceClient.fetchDataFromServerGET(BulkDataServiceClient.java:1073)
at org.waterforpeople.mapping.dataexport.service.BulkDataServiceClient.fetchDataFromServer(BulkDataServiceClient.java:986)
at org.waterforpeople.mapping.dataexport.service.BulkDataServiceClient.fetchPlacemarkData(BulkDataServiceClient.java:153)
at org.waterforpeople.mapping.dataexport.service.BulkDataServiceClient.fetchPlacemarks(BulkDataServiceClient.java:139)
at org.waterforpeople.mapping.dataexport.KMLApplet.processFile(KMLApplet.java:151)
at org.waterforpeople.mapping.dataexport.KMLApplet.executeExport(KMLApplet.java:113)
at org.waterforpeople.mapping.dataexport.KMLApplet.run(KMLApplet.java:45)
at java.lang.Thread.run(Unknown Source)

As a FLOW partner/support team member, I need a central place to access the apk for each partner organization so I can distribute them as needed.

Getting the following error when importing cleaned xls or xlsx for Test Surveys/Test Survey on Water For People instance.

Is it the importer, or is there something wrong with these files?

I exported them from the Dash, made some edits, and then attempted to import them again.

Java Plug-in 1.6.0_31
Using JRE version 1.6.0_31-b04-413-10M3623 Java HotSpot(TM) 64-Bit Server VM

User home directory = /Users/CMOfiesh

c: clear console window
f: finalize objects on finalization queue
g: garbage collect
h: display this help message
l: dump classloader list
m: print memory usage
o: trigger logging
q: hide console
r: reload policy configuration
s: dump system and deployment properties
t: dump thread list
v: dump thread stack
x: clear classloader cache

0-5: set trace level to

java.lang.StringIndexOutOfBoundsException: String index out of range: -1
at java.lang.String.substring(String.java:1931)
at java.lang.String.substring(String.java:1904)
at org.waterforpeople.mapping.dataexport.RawDataSpreadsheetImporter.executeImport(RawDataSpreadsheetImporter.java:195)
at com.gallatinsystems.framework.dataexport.applet.DataImportAppletImpl.doImport(DataImportAppletImpl.java:80)
at com.gallatinsystems.framework.dataexport.applet.DataImportAppletImpl.init(DataImportAppletImpl.java:51)
at sun.plugin2.applet.Plugin2Manager$AppletExecutionRunnable.run(Plugin2Manager.java:1639)
at java.lang.Thread.run(Thread.java:680)

testsurvey.xls
testsurvey.xlsx

develop additional logic in the importer so it can read xlsx files

As a FLOW developer, I need to make creating and updating a new FLOW instance simpler and less error-prone so that instances require less developer time and attention to manage.

This enhancement will also help to prepare for automated instance set-up and update tools that will need to be ready when we reach 20+ FLOW instances.

• assign to Dru/Chris

How to reproduce
In the IRC instance

1. Go to BKF -> AUE v2 april 1
2. the 6th item is 'Artisan Réparateur', the 7th item is 'Gestion au sein de l'AUE'. The order on the phone is the same.
3. however, the order needs to be reversed: first 'Gestion', then 'artisan'
4. using the up and down arrows in the question group list do not have an effect

Expected behaviour
Using the arrows in the question group should change the order.

WSP users in Sierra Leone weren't clearing data off the device SD cards after heavy prolonged use. As a result, they discovered that although they weren't anywhere close to the SD card capacity limits, they were hitting the limit for the number of files in a directory on their phones (since they run totally disconnected for so long and never clear thing out).

File directory limits depend on 1) the formatting of the SD Card and 2) on the size of the filename (for FAT & FAT32 filesystems [the default]) so it's not a hard limit. Based on our naming convention, though, they should be able to get over 2,000 files in the directory unless it's FAT, then, since we're writing to the root directory, it's more like 500.

This happened with WSP in Sierra Leone in March 2012, who were using FLOW in a mostly offline situation (not submitting data at the time of collection).

Modify apk signature to eliminate signature conflicts that make installation more difficult

Certain tools are too complex for users to understand, and so we will put them in a menu called Admin Only, or Admin Tools.

Contents:

Import Survey Spreadsheet
Import GIS Shapefile
new zip to xls tool (that excludes the server)
probably others, pending review

Make it so that the Admin Only/Admin Tools menu option would only be visible to user with a special permission in the datastore (so you'd have to go into the datastore admin and update the user record manually to be able to see it) ... that way the options are still there for us if we need them but are invisible to normal users (even if they're Admins).

Testing out the issue tracker.

Export of both Raw Data Report and Comprehensive report for WFP 2012 Surveys/Uganda Waterpoint 2012 is returning a 500 error when loading instances.

Tried another survey (Test Surveys/CMO Test Survey) and got the same 500 error.

The following error is from the export of the Raw Data Report of the Uganda Waterpoint 2012, but got the same one when trying to export other reports.

Java Plug-in 1.6.0_31
Using JRE version 1.6.0_31-b04-413-10M3623 Java HotSpot(TM) 64-Bit Server VM
User home directory = /Users/CMOfiesh

c: clear console window
f: finalize objects on finalization queue
g: garbage collect
h: display this help message
l: dump classloader list
m: print memory usage
o: trigger logging
q: hide console
r: reload policy configuration
s: dump system and deployment properties
t: dump thread list
v: dump thread stack
x: clear classloader cache

0-5: set trace level to

Java Plug-in 1.6.0_31
Using JRE version 1.6.0_31-b04-413-10M3623 Java HotSpot(TM) 64-Bit Server VM
User home directory = /Users/CMOfieshCalling: http://watermapmonitordev.appspot.com/surveyrestapi?action=listGroups&surveyId=7530475
response: {"message":null,"cursor":null,"resultCount":0,"class":"class org.waterforpeople.mapping.app.web.dto.SurveyRestResponse","code":null,"offset":0,"url":null,"dtoList":[{"order":1,"questionMap":null,"description":null,"name":"General ","path":"2012 Surveys/Uganda Water Point 2012","class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionGroupDto","code":"General ","surveyId":7530475,"displayName":"General ","keyId":7533331},{"order":2,"questionMap":null,"description":null,"name":"Water Point ","path":"2012 Surveys/Uganda Water Point 2012","class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionGroupDto","code":"Water Point ","surveyId":7530475,"displayName":"Water Point ","keyId":7533332}]}

Calling: http://watermapmonitordev.appspot.com/surveyrestapi?action=listQuestions&questionGroupId=7533331
response: {"message":null,"cursor":null,"resultCount":0,"class":"class org.waterforpeople.mapping.app.web.dto.SurveyRestResponse","code":null,"offset":0,"url":null,"dtoList":[{"translationMap":null,"allowDecimal":false,"optionContainerDto":null,"type":{"declaringClass":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType","class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType"},"tip":"","maxVal":0,"order":1,"isName":false,"questionHelpList":null,"collapseable":false,"path":"2012 Surveys/Uganda Water Point 2012/null","allowMultipleFlag":false,"immutable":false,"allowOtherFlag":false,"allowSign":false,"text":"Location of Water Point","minVal":0,"class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto","questionDependency":null,"mandatoryFlag":false,"questionGroupId":7533331,"questionTypeString":"GEO","surveyId":7530475,"displayName":"Location of Water Point","keyId":7529660},{"translationMap":null,"allowDecimal":false,"optionContainerDto":null,"type":{"declaringClass":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType","class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType"},"tip":"","maxVal":0,"order":2,"isName":false,"questionHelpList":null,"collapseable":false,"path":"2012 Surveys/Uganda Water Point 2012/null","allowMultipleFlag":false,"immutable":false,"allowOtherFlag":true,"allowSign":false,"text":"District/Region","minVal":0,"class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto","questionDependency":null,"mandatoryFlag":true,"questionGroupId":7533331,"questionTypeString":"OPTION","surveyId":7530475,"displayName":"District/Region","keyId":7535276},{"translationMap":null,"allowDecimal":false,"optionContainerDto":null,"type":{"declaringClass":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType","class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType"},"tip":"","maxVal":0,"order":3,"isName":false,"questionHelpList":null,"collapseable":true,"path":"2012 Surveys/Uganda Water Point 2012/null","allowMultipleFlag":false,"immutable":false,"allowOtherFlag":true,"allowSign":false,"text":"Sub-County/Town Council/Municipality","minVal":0,"class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto","questionDependency":null,"mandatoryFlag":true,"questionGroupId":7533331,"questionTypeString":"OPTION","surveyId":7530475,"displayName":"Sub-County/Town Council/Municipality","keyId":7533333},{"translationMap":null,"allowDecimal":false,"optionContainerDto":null,"type":{"declaringClass":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType","class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType"},"tip":"","maxVal":0,"order":4,"isName":false,"questionHelpList":null,"collapseable":true,"path":"2012 Surveys/Uganda Water Point 2012/General ","allowMultipleFlag":false,"immutable":true,"allowOtherFlag":true,"allowSign":false,"text":"Sub-County/Town Council/Muncipality","minVal":0,"class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto","questionDependency":null,"mandatoryFlag":true,"questionGroupId":7533331,"questionTypeString":"OPTION","surveyId":7530475,"displayName":"Sub-County/Town Council/Muncipality","keyId":7871048},{"translationMap":null,"allowDecimal":false,"optionContainerDto":null,"type":{"declaringClass":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType","class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType"},"tip":"","maxVal":0,"order":5,"isName":false,"questionHelpList":null,"collapseable":true,"path":"2012 Surveys/Uganda Water Point 2012/General ","allowMultipleFlag":false,"immutable":true,"allowOtherFlag":true,"allowSign":false,"text":"Sub-County/Town Council/Municipality","minVal":0,"class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto","questionDependency":null,"mandatoryFlag":true,"questionGroupId":7533331,"questionTypeString":"OPTION","surveyId":7530475,"displayName":"Sub-County/Town Council/Municipality","keyId":7874049},{"translationMap":null,"allowDecimal":false,"optionContainerDto":null,"type":{"declaringClass":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType","class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType"},"tip":"","maxVal":0,"order":6,"isName":false,"questionHelpList":null,"collapseable":true,"path":"2012 Surveys/Uganda Water Point 2012/General ","allowMultipleFlag":false,"immutable":true,"allowOtherFlag":true,"allowSign":false,"text":"Ward","minVal":0,"class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto","questionDependency":null,"mandatoryFlag":true,"questionGroupId":7533331,"questionTypeString":"OPTION","surveyId":7530475,"displayName":"Ward","keyId":7980087},{"translationMap":null,"allowDecimal":false,"optionContainerDto":null,"type":{"declaringClass":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType","class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType"},"tip":"","maxVal":0,"order":7,"isName":false,"questionHelpList":null,"collapseable":true,"path":"2012 Surveys/Uganda Water Point 2012/General ","allowMultipleFlag":false,"immutable":true,"allowOtherFlag":true,"allowSign":false,"text":"Ward","minVal":0,"class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto","questionDependency":null,"mandatoryFlag":true,"questionGroupId":7533331,"questionTypeString":"OPTION","surveyId":7530475,"displayName":"Ward","keyId":7955463},{"translationMap":null,"allowDecimal":false,"optionContainerDto":null,"type":{"declaringClass":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType","class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType"},"tip":"","maxVal":0,"order":8,"isName":false,"questionHelpList":null,"collapseable":true,"path":"2012 Surveys/Uganda Water Point 2012/General ","allowMultipleFlag":false,"immutable":true,"allowOtherFlag":false,"allowSign":false,"text":"Ward","minVal":0,"class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto","questionDependency":null,"mandatoryFlag":true,"questionGroupId":7533331,"questionTypeString":"FREE_TEXT","surveyId":7530475,"displayName":"Ward","keyId":7979115},{"translationMap":null,"allowDecimal":false,"optionContainerDto":null,"type":{"declaringClass":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType","class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType"},"tip":"","maxVal":0,"order":9,"isName":false,"questionHelpList":null,"collapseable":true,"path":"2012 Surveys/Uganda Water Point 2012/General ","allowMultipleFlag":false,"immutable":true,"allowOtherFlag":true,"allowSign":false,"text":"Parish","minVal":0,"class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto","questionDependency":null,"mandatoryFlag":true,"questionGroupId":7533331,"questionTypeString":"OPTION","surveyId":7530475,"displayName":"Parish","keyId":7963902},{"translationMap":null,"allowDecimal":false,"optionContainerDto":null,"type":{"declaringClass":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType","class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType"},"tip":"","maxVal":0,"order":10,"isName":false,"questionHelpList":null,"collapseable":true,"path":"2012 Surveys/Uganda Water Point 2012/General ","allowMultipleFlag":false,"immutable":true,"allowOtherFlag":true,"allowSign":false,"text":"Ward","minVal":0,"class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto","questionDependency":null,"mandatoryFlag":true,"questionGroupId":7533331,"questionTypeString":"OPTION","surveyId":7530475,"displayName":"Ward","keyId":7966930},{"translationMap":null,"allowDecimal":false,"optionContainerDto":null,"type":{"declaringClass":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType","class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType"},"tip":"","maxVal":0,"order":11,"isName":false,"questionHelpList":null,"collapseable":true,"path":"2012 Surveys/Uganda Water Point 2012/General ","allowMultipleFlag":false,"immutable":true,"allowOtherFlag":true,"allowSign":false,"text":"Parish","minVal":0,"class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto","questionDependency":null,"mandatoryFlag":true,"questionGroupId":7533331,"questionTypeString":"OPTION","surveyId":7530475,"displayName":"Parish","keyId":7969026},{"translationMap":null,"allowDecimal":false,"optionContainerDto":null,"type":{"declaringClass":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType","class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType"},"tip":"","maxVal":0,"order":12,"isName":false,"questionHelpList":null,"collapseable":true,"path":"2012 Surveys/Uganda Water Point 2012/General ","allowMultipleFlag":false,"immutable":true,"allowOtherFlag":true,"allowSign":false,"text":"Parish","minVal":0,"class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto","questionDependency":null,"mandatoryFlag":true,"questionGroupId":7533331,"questionTypeString":"OPTION","surveyId":7530475,"displayName":"Parish","keyId":7965960},{"translationMap":null,"allowDecimal":false,"optionContainerDto":null,"type":{"declaringClass":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType","class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType"},"tip":"","maxVal":0,"order":13,"isName":false,"questionHelpList":null,"collapseable":true,"path":"2012 Surveys/Uganda Water Point 2012/General ","allowMultipleFlag":false,"immutable":true,"allowOtherFlag":true,"allowSign":false,"text":"Ward","minVal":0,"class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto","questionDependency":null,"mandatoryFlag":true,"questionGroupId":7533331,"questionTypeString":"OPTION","surveyId":7530475
Trace message truncated for length over 10K

Calling: http://watermapmonitordev.appspot.com/surveyrestapi?action=listQuestions&questionGroupId=7533332
response: {"message":null,"cursor":null,"resultCount":0,"class":"class org.waterforpeople.mapping.app.web.dto.SurveyRestResponse","code":null,"offset":0,"url":null,"dtoList":[{"translationMap":null,"allowDecimal":false,"optionContainerDto":null,"type":{"declaringClass":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType","class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType"},"tip":"","maxVal":0,"order":1,"isName":false,"questionHelpList":null,"collapseable":false,"path":"2012 Surveys/Uganda Water Point 2012/null","allowMultipleFlag":false,"immutable":false,"allowOtherFlag":false,"allowSign":false,"text":"Photo of Water Point","minVal":0,"class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto","questionDependency":null,"mandatoryFlag":false,"questionGroupId":7533332,"questionTypeString":"PHOTO","surveyId":7530475,"displayName":"Photo of Water Point","keyId":7530477},{"translationMap":null,"allowDecimal":false,"optionContainerDto":null,"type":{"declaringClass":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType","class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType"},"tip":"Observational Question -- Enumerator should determine if the source is improved or unimproved according to local guidelines. ","maxVal":0,"order":2,"isName":false,"questionHelpList":null,"collapseable":false,"path":"2012 Surveys/Uganda Water Point 2012/null","allowMultipleFlag":false,"immutable":false,"allowOtherFlag":false,"allowSign":false,"text":"Is this an improved water source/system?","minVal":0,"class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto","questionDependency":null,"mandatoryFlag":true,"questionGroupId":7533332,"questionTypeString":"OPTION","surveyId":7530475,"displayName":"Is this an improved water source/system?","keyId":7532362},{"translationMap":null,"allowDecimal":false,"optionContainerDto":null,"type":{"declaringClass":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType","class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType"},"tip":"Observational Question -- Enumerator should observe the water point and determine what type it is. ","maxVal":0,"order":3,"isName":false,"questionHelpList":null,"collapseable":false,"path":"2012 Surveys/Uganda Water Point 2012/null","allowMultipleFlag":false,"immutable":false,"allowOtherFlag":true,"allowSign":false,"text":"What is the type of the water point?","minVal":0,"class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto","questionDependency":null,"mandatoryFlag":true,"questionGroupId":7533332,"questionTypeString":"OPTION","surveyId":7530475,"displayName":"What is the type of the water point?","keyId":7529664},{"translationMap":null,"allowDecimal":false,"optionContainerDto":null,"type":{"declaringClass":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType","class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType"},"tip":"Observational Question -- Enumerator should observe the water point and determine what type it is.","maxVal":0,"order":4,"isName":false,"questionHelpList":null,"collapseable":false,"path":"2012 Surveys/Uganda Water Point 2012/null","allowMultipleFlag":false,"immutable":false,"allowOtherFlag":true,"allowSign":false,"text":"What is the type of this water point?","minVal":0,"class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto","questionDependency":null,"mandatoryFlag":true,"questionGroupId":7533332,"questionTypeString":"OPTION","surveyId":7530475,"displayName":"What is the type of this water point?","keyId":7532365},{"translationMap":null,"allowDecimal":false,"optionContainerDto":null,"type":{"declaringClass":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType","class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType"},"tip":"If an exact number is not know or if it varies provide an estimate.","maxVal":0,"order":5,"isName":false,"questionHelpList":null,"collapseable":false,"path":"2012 Surveys/Uganda Water Point 2012/null","allowMultipleFlag":false,"immutable":false,"allowOtherFlag":false,"allowSign":false,"text":"How many people use this water source?","minVal":0,"class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto","questionDependency":null,"mandatoryFlag":true,"questionGroupId":7533332,"questionTypeString":"NUMBER","surveyId":7530475,"displayName":"How many people use this water source?","keyId":7530480},{"translationMap":null,"allowDecimal":false,"optionContainerDto":null,"type":{"declaringClass":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType","class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType"},"tip":"","maxVal":0,"order":6,"isName":false,"questionHelpList":null,"collapseable":false,"path":"2012 Surveys/Uganda Water Point 2012/null","allowMultipleFlag":false,"immutable":false,"allowOtherFlag":false,"allowSign":false,"text":"Is there anyone in the community that does not have access to an improved water point?","minVal":0,"class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto","questionDependency":null,"mandatoryFlag":true,"questionGroupId":7533332,"questionTypeString":"OPTION","surveyId":7530475,"displayName":"Is there anyone in the community that does not have access to an improved water point?","keyId":7528887},{"translationMap":null,"allowDecimal":false,"optionContainerDto":null,"type":{"declaringClass":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType","class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType"},"tip":"","maxVal":0,"order":7,"isName":false,"questionHelpList":null,"collapseable":false,"path":"2012 Surveys/Uganda Water Point 2012/null","allowMultipleFlag":false,"immutable":false,"allowOtherFlag":false,"allowSign":false,"text":"How many people do not have access to the improved water system?","minVal":0,"class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto","questionDependency":null,"mandatoryFlag":false,"questionGroupId":7533332,"questionTypeString":"NUMBER","surveyId":7530475,"displayName":"How many people do not have access to the improved water system?","keyId":7529671},{"translationMap":null,"allowDecimal":false,"optionContainerDto":null,"type":{"declaringClass":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType","class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType"},"tip":"","maxVal":0,"order":8,"isName":false,"questionHelpList":null,"collapseable":false,"path":"2012 Surveys/Uganda Water Point 2012/null","allowMultipleFlag":true,"immutable":false,"allowOtherFlag":true,"allowSign":false,"text":"Why doesn't everyone have access to the improved water system?","minVal":0,"class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto","questionDependency":null,"mandatoryFlag":true,"questionGroupId":7533332,"questionTypeString":"OPTION","surveyId":7530475,"displayName":"Why doesn't everyone have access to the improved water system?","keyId":7533340},{"translationMap":null,"allowDecimal":false,"optionContainerDto":null,"type":{"declaringClass":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType","class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType"},"tip":"If the exact year is not known provide an estimate. ","maxVal":0,"order":9,"isName":false,"questionHelpList":null,"collapseable":false,"path":"2012 Surveys/Uganda Water Point 2012/null","allowMultipleFlag":false,"immutable":false,"allowOtherFlag":false,"allowSign":false,"text":"What year was the water point/system constructed or rehabilitated?","minVal":0,"class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto","questionDependency":null,"mandatoryFlag":true,"questionGroupId":7533332,"questionTypeString":"NUMBER","surveyId":7530475,"displayName":"What year was the water point/system constructed or rehabilitated?","keyId":7529673},{"translationMap":null,"allowDecimal":false,"optionContainerDto":null,"type":{"declaringClass":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType","class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType"},"tip":"If the community member does not know Water For People staff will know.","maxVal":0,"order":10,"isName":false,"questionHelpList":null,"collapseable":false,"path":"2012 Surveys/Uganda Water Point 2012/null","allowMultipleFlag":false,"immutable":false,"allowOtherFlag":false,"allowSign":false,"text":"Is the water point part of a Water For People supported project?","minVal":0,"class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto","questionDependency":null,"mandatoryFlag":true,"questionGroupId":7533332,"questionTypeString":"OPTION","surveyId":7530475,"displayName":"Is the water point part of a Water For People supported project?","keyId":7529674},{"translationMap":null,"allowDecimal":false,"optionContainerDto":null,"type":{"declaringClass":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType","class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType"},"tip":"","maxVal":0,"order":11,"isName":false,"questionHelpList":null,"collapseable":false,"path":"2012 Surveys/Uganda Water Point 2012/null","allowMultipleFlag":true,"immutable":false,"allowOtherFlag":true,"allowSign":false,"text":"What role did Water For People play?","minVal":0,"class":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto","questionDependency":null,"mandatoryFlag":true,"questionGroupId":7533332,"questionTypeString":"OPTION","surveyId":7530475,"displayName":"What role did Water For People play?","keyId":7530482},{"translationMap":null,"allowDecimal":false,"optionContainerDto":null,"type":{"declaringClass":"class org.waterforpeople.mapping.app.gwt.client.survey.QuestionDto$QuestionType","class":"class org.waterforpeople.mapping.app.gwt.c
Trace message truncated for length over 10K

Calling: http://watermapmonitordev.appspot.com/databackout?action=listInstance&surveyId=7530475&includeDate=true
java.io.IOException: Server returned HTTP response code: 500 for URL: http://watermapmonitordev.appspot.com/databackout?action=listInstance&surveyId=7530475&includeDate=true
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1436)
at org.waterforpeople.mapping.dataexport.service.BulkDataServiceClient.fetchDataFromServerGET(BulkDataServiceClient.java:1073)
at org.waterforpeople.mapping.dataexport.service.BulkDataServiceClient.fetchDataFromServer(BulkDataServiceClient.java:986)
at org.waterforpeople.mapping.dataexport.service.BulkDataServiceClient.fetchInstanceIds(BulkDataServiceClient.java:283)
at org.waterforpeople.mapping.dataexport.GraphicalSurveySummaryExporter.fetchAndWriteRawData(GraphicalSurveySummaryExporter.java:348)
at org.waterforpeople.mapping.dataexport.GraphicalSurveySummaryExporter.export(GraphicalSurveySummaryExporter.java:268)
at com.gallatinsystems.framework.dataexport.applet.DataExportAppletImpl.doExport(DataExportAppletImpl.java:57)
at com.gallatinsystems.framework.dataexport.applet.DataExportAppletImpl.init(DataExportAppletImpl.java:31)
at sun.plugin2.applet.Plugin2Manager$AppletExecutionRunnable.run(Plugin2Manager.java:1639)
at java.lang.Thread.run(Thread.java:680)

When importing or exporting files to the Dashboard (xls, kmz), Overall Progress windows that display status do not go to completion numerically, even though Export Complete and Close buttons appear.

This is confusing for users, especially when Java Console is not enabled.

Screenshots:
Progress Window on export
Progress Window on import

• Device listing in System Summary
• Device listing in Create and Edit Assignment pages

Related to progress on Issue #49

How to reproduce
In the IRC instance, reordering of question groups hangs the interface.

1. Click on survey group 'BKF', survey 'AUE v.2 april 1'.
2. try to move 'Artisan Réparateur' down using the green arrow button
3. a pop-up 'Saving, please wait' appears, but nothing further happens and the interface stops responding

They have mapped the attributes for the survey. I also tried running remap to access point once today (13 April). I see all the data in the raw data manager and access point manager.

The are no data for Uganda on the public facing map (http://watermapmonitordev.appspot.com/). I notice there is no listing of Uganda in the countries listing on that page. But I see Uganda in the datastore.

I ran a kmz on 11 April and also do not see Uganda data in that file.

WFP kmz from 11 April

Create some initial test scripts for the APK, esp. for the new features and any historic pain points (parts of the app workflow that have caused issues when testing or unexpected breaks). See the testing spreadsheet for a list of features: https://docs.google.com/spreadsheet/ccc?key=0AgEEfk7lVujkdGx2eTBScnp3MndIV0FwOFRnV3ZNbWc#gid=1

The APK sometimes hangs when a picture is taken. Amit has observed this both on a Motorola Milestone (India WFP field team) and on a Samsung Galaxy S (his own). The observation of Amit:

"The moment I click on take picture, it takes a snapshot and then the option comes as save, and when I click on save, the system blacks out. I get a alert that I have to force quit. However, good thing is that after force quit, when I again start using FLOW, the previous data remains intact."

Dashboard feature to host system messages about APK to alert users, could direct users to a hyperlink that connects to s3 (easy way, 1.5); long term make it available within the dashboard via a hardcoded link.

Can put this in the admin only menu to generate link.