I'm +1 for improving the ReadMe to encourage users to use the package. I think that the network graphs of Atlanta would be a nice visualization to add in the ReadMe right after the inference example. Aso an explanation of the networks based segregation in the readme would be cool (or perhaps only mentioning the notebook, but I think a summary would be suitable).

for aspatial_examples.ipynb

sacramentot2.shp is not a file in any installed dataset.

Why is the w being passed to manhattan here?
Shouldn't this be the shimbel matrix (i.e., shortest path for each i,j)?

the following will result in a np.nan value for the statistic because n_nonhsip_black_persons has NaN values.

dc = Community(source='ltdb', cbsafips='47900')
dc = dc.tracts.merge(dc.census,left_on='geoid', right_index=True)
dc_sd = SpatialDissim(dc, group_pop_var='n_nonhisp_black_persons', total_pop_var='n_total_pop')

We should either move to more robust numpy operators that handle nans, or check whether there are any present in group_pop_var or total_pop_var and raise accordingly

As per pysal/libpysal#178 and pysal/libpysal#181

from: http://pysal.readthedocs.io/en/latest/developers/index.html
to: https://github.com/pysal/pysal/wiki

It is probably better to change the SxPx and SxPy to 'distance-decay' isolation/exposure as this could be confused with the spatial isolation and spatial exposure of https://onlinelibrary.wiley.com/doi/abs/10.1111/j.0081-1750.2004.00150.x

Although this is mentioned in the docstrings notes of these indexes.

• Some indexes such as Modified Dissimilarity (Dct), Modified Gini (Gct) and Bias-Corrected Dissimilarity (Dbc) could be leveraged to work in parallel since they rely on independent draws of probability distributions and recalculating the index.

• Also, the inference wrappers (Infer_Segregation and Compare_Segregation) could be leveraged to work in parallel since they rely on independent simulations framework.

A possibility to implement is to use Dask (https://github.com/dask/dask), concurrent.futures (https://docs.python.org/3/library/concurrent.futures.html), etc.

Hi, I've been getting an error when I try to install segregation using option 2, the pip install option.

Command "python setup.py egg_info" failed with error code 1 in C:\Users\lm13n\AppData\Local\Temp\pip-install-r6y591a0\fiona\

I am installing into an Anaconda Python 3.6.8 environment on Windows 10. However, option 1 installs successfully.

Thanks,
Levon

Currently, we are point to https://github.com/spatialucr/geosnap/tree/master/geosnap/data, but we need to update this.

I'm running into this error message when trying to use the Inference wrappers and the function SingleValueTest. Any suggestions? Thanks!

Compare segregation with giddy.

Hello I am just opening this issue following a discussion with @darribas to contribute into segregation the statistic proposed in the following paper:
(Olteanu and al. 2019)
I have a working version in another repository here I and I just wanted to kick off the discussion on how to integrate it in, and have your advice on the process.

Currently nbsit and Multi_Information_Theory function are performing the same calculations. I think it makes more sense to go with the current segregation API, i.e., the latter approach:

I could make a PR with this change, but I still didn't manage to fetch 5000 buffer for the example.

I'm getting this ModuleNotFoundError:

I have segregation (1.1.1) and PySAL 2.1.0 in my environment.

Any suggestions? Thanks!

Following what was discussed in #73 (comment), the idea is to create a function that would only calculate a set of functions given by the user, rather than all the measures that could take more time.

most of the plotting functionality started as a convenience for ourselves, but now that we're using them more often we need to make sure we're returning at least the axes object so users can edit/save/combine plots, etc

• network
• profile
• util

(Only the first warning is relevant to this issue).

DistanceDecayExposure for example, expects wgs84. We decided against implicit reprojection, but we could do a check and raise a warning if we cant determine the crs is wgs84

When running segregation indexes, I get a non-corresponding output legend (Out). It repeats the same output line: "segregation.spatial.spatial_indexes.SpatialDissim" regardless if the index has change... here an example of two different indexes, same Out legend:

This seems to be the default Output line in my local results... in every index...but this is not the case in the examples from the notebook.

Any ideas as to what can be the problem in my local version? Thanks!!!

i turned on the zenodo integration for this repo, so the next github release should get tagged automatically with a DOI. Once that happens, we should add the badge to the repo

To Do:

• cut github release
• add badge to readme

The results are the same, this is just an enhancement to clean the code of Compare_Segregation.

ps.: in the decomposition framework, this is used.

Consider implement the multi-group indexes. A good start can be the following (Luc Anselin questioned that to me at the AAG 2019):
"Reardon, Sean F., and Glenn Firebaugh. "Measures of multigroup segregation." Sociological methodology 32.1 (2002): 33-67."

@renanxcortes and I have had some conversations about refactoring this project so that, instead of each segregation index being implemented as its own class, we might have two classes (e.g. spatial and aspatial), and the indices themselves would move to subclasses or functions.

In another thread, as we work on spopt we're making a concerted effort to use the BaseClass/Mixin structure that scikit-learn uses. I want to raise the option here of adopting a similar architecture, which i think would make a lot of sense for this project

One nice index to implement in the module is the one proposed in:
https://amstat.tandfonline.com/doi/full/10.1080/07350015.2012.707586

Some previous work has already been done in that sense as Rathelot kindly shared with me original R code of this index and I'll put here more details in the future.

Requirements of segregation should explicitly ask for a version of scikit-learn that has haversine_distance distance measures. Otherwise, it will raise an error while importing.

Since our paper "Comparative Spatial Segregation Analytics" was accepted on Spatial Demography ( 👍 👍 👍 ) we can update the cell that says:

Let's decompose these difference according to Rey, S. et al "Comparative Spatial Segregation Analytics". Forthcoming. You can check the options available in this decomposition below:

In this notebook:
https://github.com/pysal/segregation/blob/master/notebooks/decomposition_wrapper_example.ipynb

compute_segregation_profile calculates SIT for varying distances, but it should be extended to calculate any index that takes a W or a Network

currently, these include SIT or spatial divergence, but this might be a good time to think through which others could be re/written to follow the subclass pattern those use (I think spatialdissim, maybe others). That would help get us on the road to #4

sidenote that I still think compute_segregation_profile is a bit verbose, so i'd be open to new name suggestions

these print statements dont show up in jupyterlab for me so I end up with a ton of blank space in the notebook and no indication of what's happening.

What about shifting over to tqdm.auto which would be a bit more aesthetically pleasing and robust?

I've been struggling with the concentration indexes (especially the ACO), because I'm trying to match the values with the ones that is generated by OasisR (https://cran.r-project.org/web/packages/OasisR/OasisR.pdf)... in the original paper of OasisR the author (page 12, Table 6) states that this index matches the one of the GSA implementations.

However, I'm closely checking line by line of the functions and am not seeing any difference of the R implementation (https://github.com/cran/OasisR/blob/99f5d028c205329c4f3b1355e5bcaa09e1fcc077/R/SegFunctions.R#L1358) and our implementation. I highlight that the R and GSA implementation might be not correct, but I wanted to open a discussion of this.

The original formula is:

To reproduce this in Python (edit path needed):

import geopandas as gpd
import segregation
from segregation.spatial import Absolute_Concentration

irreg = gpd.read_file('C:\\Users\\renan\\Desktop\\oasisTests\\irregular_lattice_50.shp')
irreg['group_pop_var'] = list(range(1, 51))
irreg['total_pop_var'] = 100
index1 = Absolute_Concentration(irreg, 'group_pop_var', 'total_pop_var')
index1.statistic

to reproduce this in R (edit path needed):

library(OasisR)
library(rgdal)
irreg<-readOGR("C:\\Users\\renan\\Desktop\\oasisTests","irregular_lattice_50")
vector1 <- seq(1,50) # Group 1 Population
tot <- rep(100, 50)
vector2 <- tot - vector1 # Group 2 Population
irreg_input_data <- cbind(vector1, vector2)
ACO(irreg_input_data, spatobj = irreg)
# The first value is the ACO value for the first group

The irregular lattice used is attached.
irregular_lattice_50.zip

following the same model as geosnap/the other pysal packages

in this piece, Reardon et al calculate multiscalar segregation profiles by calculating several spatial information theory indices with different specifications of W (increasing in kernel distance). By calculating the ratio of SIT w/ a small-distance W against SIT w/ a large-distance W, they can decompose segregation as a function of scale

"In particular, we find that the proportion of micro-scale segregation that is due to macro-scale segregation ranges between 20% and 80% across these 40 metropolitan areas, with macro-scale segregation generally accounting for a larger share of white-black segregation than of white-Hispanic or white Asian segregation. This heterogeneity raises a number of >additional questions that we (and other scholars, we hope) will address in subsequent research."

Now that we have SIT in the library, we could provide a function to do the same. This would also serve to help movtivate the use of network-based kernel weights, like discussed here

Following the bottom of readme of https://github.com/pysal/spaghetti.

Described in https://doi.org/10.1111/j.1538-4632.2007.00699.x

here and below the columns are hardcoded with strings but they should be variables

This is a low-hanging fruit to include the bootstrap approach that we discussed several times. I've seen now that https://www.jstatsoft.org/article/view/v089i07 also implemented it for single values. This is something that I think it is worth doing and relatively easy.

given in Echenique & Fryer, 2005

I gave it a placeholder called "dual_composition" but that's not a very accurate name

The definition of urban core is essential to the centralization indices. Right now, the urban core is defined as the geometric center of the centroids of input polygons (e.g. tract boundaries). While this definition is convenient and could be meaningful in some cases, it might not be the best way to define urban core in others. Some other options could be:

• the central business district (CBD)
• the population centroid
• the centroid of the MSA (the dissolved boundary of all tracts)

I think it would be nice to provide these options to users.

Another complication related to measuring the centralization dimension of a MSA is that it could host several cities, each of which has an urban core.

TODO Prior to next release:

Critical

• RX: rename profile to compute_all()
• RX: compute_all should return a pandas.DataFrame instead of a dict
• RX: smart counterfactual approach
• RX: decide on distance decay isolation name
• EK: docstrings for network funcs
• EK: dosctrings for util plots
• EK: add tests for network measures
• EK: move udst dependencies into extra_requirements.txt and add warning to network functions
• EK: update readme
  • add DOI badge to readme
  • add network example

Minor:

• decide on signature for spatial indices as discussed here
  e.g.:

def localize(data, w):
	new_data = []
	w = pysal.weights.insert_diagonal(w) # attach focal unit to environment
	for y in data:
		new_data.append(lag_spatial(w,y)
	return new_data

from Reardon & O'Sullivan 2004

as described by Reardon et al

I think that after adding tqdm in #109, TravisCI broke and the error log is:

Using /home/travis/build/pysal/segregation/miniconda/envs/test-env/lib/python3.7/site-packages
Finished processing dependencies for segregation==1.1.0
The command "python setup.py install" exited with 0.
21.95s$ nosetests -v segregation --with-coverage
Failure: ImportError (libcfitsio.so.5: cannot open shared object file: No such file or directory) ... ERROR
======================================================================
ERROR: Failure: ImportError (libcfitsio.so.5: cannot open shared object file: No such file or directory)
----------------------------------------------------------------------

Not really sure how to to debug this. Any ideas?

Segregation on PySAL started to fail on travis https://travis-ci.org/pysal/pysal/jobs/566597373#L2405

This is due to indexes that rely on simulations and, therefore, numpy seeds. Travis now is building using numpy 1.17.0 (released 6 days ago). Some features were changed in the generating process of the seeds as you can read in https://github.com/numpy/numpy/releases.

I upgraded locally my numpy to 1.17.0 (previously it was 1.16.0) and, now, my seed generates the values that match the new tests (7 digit precision).

Since I don't think it is a good thing the rely on the numpy version for the simulation-based tests... I think I'll increase the degree of tolerance of all tests and rewrite the tests (inference based also will change since they also rely on numpy seeds).

tests are failing because they use the sacramento data and havent been updated for the new libpysal api

The sentence To see the estimated D in the first generic example above, the user would have just to run index.statistic to see the fitted value. should change its location to after the "d_index" call and be updated to "d_index".

Also, we can pt a point after "total_population" and put uppercase "A" in:

"a typical call would be something like this:"