aiandglobaldevelopmentlab / eo-poverty-review Goto Github PK

@ksakamoto09 I have the following major comments and some minor in text.

• 1. The Review results, qualitatively, is not coming through. One reason is that we have yet not developed a proper categorization. Also, we should list our findings in the appendix at least, with their categories.
• 2. The Schema (or categorization) that order the review is still missing. Again, the Lundberg et al paper is what we need.
• 3. Not sure we need the tables in the appendix. They are just a list of keywords. I suggest removing them.
• 4. No clear connection between Motivation and Introduction and the section that follows (Classical Geography).
• 5. The difference between Section 4 and 5 needs clarification. Both are about EO and so the reader will likely confuse them. My recommendation is to have a Background section where we talk about spatial stats in one subsection and then a subsection of recent trends of using EO for measurement but not causality. Then we introduce our Review in section 5 where we discuss the finding of the literature review and the categorization. Here, we define estimands usual estimands (e.g., from ATE, to CATE, to some local average treatments, geo-RDD style), assumptions (e.g., relaxation of SUTVA).
• 6. When the reader starts reading section 5, it is actually unclear that they are reading the results of the review. This section reads like the rest. That is not what we want.
• 7. Figure 3, for EO-ML causal workflow would be an important contribution.
• 8. Bake together Conclusion with Discussion.
• 9. I am not sure Figure 1 adds much right now. It takes a lot of space with not much added value.

Can you also address previous feedback #2

@ksakamoto09 Here is a GitHub where we can put supplementary information for the review.

The following are some highlevel comments, @ksakamoto09

• 1. We can make the opening paragraphs stronger. It is a little generic right now. We should create a gap and relevance statement in the first paragraph by clarifying more exactly where geography of poverty may benefit from EO-ML powered causal inference. IT should be making clear why a review is relevant. For that, we can emphasize that the two streams of literature (geo of poverty) and EO-ML has up to recently evolved independently. Thus, the literature needs more information where there are synergies. This review contributes towards that.
• 2. In terms of section 2.5, see Lundberg et al (See attached) for summarizing methods in tables, especially if we want to include estimators. It should be made clear why the reader should think about these estimators.
• 3. We should also discuss common estimands .
• 4. As previously discussed, I am not sure we want to conduct a quantitative literature survey (section 5). Although some of the summaries are quantitative, we want to conduct a qualitative analysis of key findings in the literature. This goes back to defining a schema. While we cannot really summarize all papers, we want to highlight trends and insights based on our 46 paper and 24 preprints. That should be the core foundational results of the paper, which underpings our main contribution – future direction, gaps, and open research questions.
• 5. Generally, the paper has too many subsection with short paragraphs. Either we want to build out those section with deeper discdussions or reduce the nr of sections.
• 6. See minor comments in the manuscript.

Lundberg et al_2021_What Is Your Estimand.pdf