carl24k / fight-churn Goto Github PK

In listing 2.3 activity churn is calculated. But this isn't exactly according to the stated definition. Take this example:
a = activity
--Month 1 --Month 2
|a--------|||--------a|

If a user has activity in the beginning of the start period and then nothing until the end of the end period, it will have past more than the period considered inactive (1 month). But the user is not considered churned. I guess it sort of makes sense since the user is active again, but this listing doesn't really fulfill the definition.

Hello, I'm currently using Python 3.6.8, and I'm having trouble installing the packages listed in my requirements.txt file. When I run pip install -r requirements.txt, it's not able to find the latest version of many of the packages. To get around this, I've been manually editing the requirements.txt file to specify the versions that are available, but there are a lot of packages with incorrect version numbers.

I've also tried Googling this issue, but I'm not sure why my package manager isn't able to find the latest versions of these packages. Do you have any suggestions for how I can fix this?

Thank you for your help.

#8 23.79 ERROR: Could not find a version that satisfies the requirement scipy==1.6.2 (from versions: 0.8.0, 0.9.0, 0.10.0, 0.10.1, 0.11.0, 0.12.0, 0.12.1, 0.13.0, 0.13.1, 0.13.2, 0.13.3, 0.14.0, 0.14.1, 0.15.0, 0.15.1, 0.16.0, 0.16.1, 0.17.0, 0.17.1, 0.18.0, 0.18.1, 0.19.0, 0.19.1, 1.0.0, 1.0.1, 1.1.0, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.4.0, 1.4.1, 1.5.0, 1.5.1, 1.5.2, 1.5.3, 1.5.4)
#8 23.79 ERROR: No matching distribution found for scipy==1.6.2
------

#8 6.064 ERROR: Could not find a version that satisfies the requirement joblib==1.2.0 (from versions: 0.3.2d.dev, 0.3.2e.dev, 0.3.2f.dev, 0.3.2g.dev, 0.7.0d, 0.1a0.dev0, 0.2a0.dev0, 0.3a0.dev0, 0.3.1a0.dev0, 0.3.2.dev0, 0.3.2a0.dev0, 0.3.2b0.dev0, 0.3.2rc0.dev0, 0.3.3a0.dev0, 0.3.3b0.dev0, 0.3.3rc0.dev0, 0.3.4.dev0, 0.3.5.dev0, 0.3.6.dev0, 0.3.7.dev0, 0.4.0.dev0, 0.4.1.dev0, 0.4.2.dev0, 0.4.3.dev0, 0.4.4.dev0, 0.4.5.dev0, 0.4.6.dev0, 0.5.0.dev0, 0.5.0a0.dev0, 0.5.1.dev0, 0.5.2.dev0, 0.5.3.dev0, 0.5.4.dev0, 0.5.5.dev0, 0.5.6.dev0, 0.5.7.dev0, 0.5.7a0.dev0, 0.5.7b0.dev0, 0.5.7, 0.6.0a0, 0.6.0b0, 0.6.0b2, 0.6.0b3, 0.6.0, 0.6.1, 0.6.2, 0.6.3, 0.6.4, 0.6.5, 0.7.0a0, 0.7.0b0, 0.7.0rc0, 0.7.1, 0.8.0a0, 0.8.0a2, 0.8.0a3, 0.8.0, 0.8.1, 0.8.2, 0.8.3, 0.8.3.post1, 0.8.4, 0.9.0b2, 0.9.0b3, 0.9.0b4, 0.9.1, 0.9.2, 0.9.3, 0.9.4, 0.10.0, 0.10.2, 0.10.3, 0.11a3, 0.11, 0.12.0, 0.12.1, 0.12.2, 0.12.3, 0.12.4, 0.12.5, 0.13.0, 0.13.1, 0.13.2, 0.14.0, 0.14.1, 0.15.0, 0.15.1, 0.16.0, 0.17.0, 1.0.0, 1.0.1, 1.1.0a0, 1.1.0, 1.1.1)

bfasilva: Hello Carl, hope you had a nice weekend. Just through your github repo and run the installation. 3 notes that may be useful to change to make the default values work seamless:
bfasilva: 1) The requirements.txt does not play well with the latest Python version (3.9). After installing 3.7 and using it for the project, it worked like a charm
bfasilva: 2) On metric_framework/conf the configuration file socialnet6_metrics.json can be renamed socialnet7_metrics.json
bfasilva: 3) Inside observe_churn.py, defaults starting on line 44 could be changed to: line
bfasilva: schema: churnsim2 -> socialnet7,
bfasilva: frdt: 2019-03-04 -> 2020-01-01
bfasilva: tode: 2019-05-06 -> 2020-01-08
bfasilva: The framework is looking great, thanks!
bfasilva: Have a nice week!

I can not create schema (https://github.com/carl24k/fight-churn#createdb 1.3.1 Schema Creation)
It gives error:

I'm not sure if this is intended, but the path for output files (e.g. CSV) are outside of the repo folder (fight-churn), if the ../../../ relative path is used. If this is expected behavior, could you please help me understand the benefit of this approach? Thank you!

Hi Carl,

I am having issues with installing the DB Schema in PyCharm.

`C:\Users\alberto.ordigoni\Downloads\fight-churn-master\venv\Scripts\python.exe C:/Users/alberto.ordigoni/Downloads/fight-churn-master/fight-churn-master/fightchurn/datagen/churndb.py
Traceback (most recent call last):
File "C:\Users\alberto.ordigoni\Downloads\fight-churn-master\fight-churn-master\fightchurn\datagen\churndb.py", line 47, in
setup_churn_db(schema_name)
File "C:\Users\alberto.ordigoni\Downloads\fight-churn-master\fight-churn-master\fightchurn\datagen\churndb.py", line 27, in setup_churn_db
con_string = f"postgresql://{os.environ['CHURN_DB_HOST']}/{os.environ['CHURN_DB']}?user={os.environ['CHURN_DB_USER']}&password={os.environ['CHURN_DB_PASS']}"
File "C:\Users\alberto.ordigoni\AppData\Local\Programs\Python\Python310\lib\os.py", line 679, in getitem
raise KeyError(key) from None
KeyError: 'CHURN_DB_HOST'

Process finished with exit code 1`

Could you please assist?

Thanks,
Alberto

The README says the project is MIT licensed, and to refer to LICENSE.md. However, the repository doesn't seem to have a LICENSE.md file at the root level. Please remember to add this file :-)

Hi, I'm doing data-generation using fight-churn github.

I have a trouble with running churnsim.py
The screen below is an error message that comes out when I run churnsim.py.

How can I fix it?
(p.s I barely ran a churndb.py and created a schema.)

run churnsim.py

TypeError Traceback (most recent call last)
~\Pictures\fight-churn\fightchurn\datagen\churnsim.py in
246
247
--> 248 run_churn_simulation(model_name, start_date, end_date, init_customers)

~\Pictures\fight-churn\fightchurn\datagen\churnsim.py in run_churn_simulation(model_name, start_date, end_date, init_customers, random_seed)
232 if random_seed is not None:
233 random.seed(random_seed) # for random
--> 234 churn_sim = ChurnSimulation(model_name, start_date, end_date, init_customers,random_seed)
235 churn_sim.run_simulation()
236

~\Pictures\fight-churn\fightchurn\datagen\churnsim.py in init(self, model, start, end, init_customers, seed)
45 if version in ('utility','population','country'):
46 continue
---> 47 behave_mod=FatTailledBehaviorModel(self.model_name,seed,version)
48 self.behavior_models[behave_mod.version]=behave_mod
49 self.model_list.append(behave_mod)

~\Pictures\fight-churn\fightchurn\datagen\behavior.py in init(self, name, random_seed, version)
134 self.exp_fun = lambda x: np.power(self.exp_base,x)
135
--> 136 super(FatTailledBehaviorModel,self).init(name,random_seed,version)
137
138 def scale_correlation_to_covariance(self):

~\Pictures\fight-churn\fightchurn\datagen\behavior.py in init(self, name, random_seed, version)
91
92 # Save to a csv
---> 93 save_path = os.getenv('CHURN_OUT_DIR') + name + '/'
94 os.makedirs(save_path, exist_ok=True)
95 copy_path = save_path + name + '_simulation_model.csv'

TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'

I ran into an issue in when running listing 3.7, specifically PyCharm could not find the the module listing_3_7_metric_qa_plot.py.
This was easily fixed by marking the listings/chap3 directory as sources root, but this directory did not appear in the list of directories in section 1.2.2.1 of the readme.

Hi Carl!

I recently bought your book and loving it so far, however I am having issues setting the relevant data. Currently at the Jupyter Notebook Setup stage and I am running into this error

I am implementing the metrics taught in the book, and I am very satisfied with the fact that the book covers complex cases, through which my company goes. However, although mentioned in the book, I think it does not teach how to deal with subscriptions that start with an initial discount, but the full price will be charged in the next renewal cycle. The problem that I see here is that the MRR changes for the same subscription and the net retention calculus would be compromised.

In my opinion, the way to overcome this issue is to split the subscription into several subscriptions that have their own MRR. Something like this:

Initial example:

• subscription_id: 1
• account_id: 1
• product_id: 1
• start_date: 2021-10-01
• end_date: None (still active)
• mrr: 10 (with discount) but full price is 20

After split:

First subscription:

• subscription_id: 1_1
• account_id: 1
• product_id: 1
• start_date: 2021-10-01
• end_date: 2021-11-01
• mrr: 10

Second subscription:

• subscription_id: 1_2
• account_id: 1
• product_id: 1
• start_date: 2021-11-01
• end_date: None (still active)
• mrr: 20

Is this the best solution for this problem?

Hi,

I am trying to run the churn listing function. I successfully installed fightchurn package in a virtual environment. After that I can also import run_churn_listing. But when I run set_churn_environment it is not able to find the function. Am I running it the right way? Adding screenshot for reference.

When I try executing the ./run_churnsim.sh from terminal within pycharm, following the setup procedures, I get a lot of errors towards the end. All of these errors shout out, "ModuleNotFoundError: No module named 'tkinter'". This package is not there in the requirements.txt.

Even when I tried to install this package from the pycharm's terminal like, pip install tkinter, I get the error like below:

Collecting tkinter
  Could not find a version that satisfies the requirement tkinter (from versions: )
No matching distribution found for tkinter
(venv) 

Developer Environment Details:

• Ubuntu OS
• venv setup per guidelines
• Python version is 3.7.5

This is a show stopper for trying out the code, isn't it? :(

Hi Carl,

I really like your book and enjoy the content that you put together.

While running listing 8.1, I came across an error that I found not locating the cause. I was running the code in the Jupyter notebook environment.

Below scripts are successfully run.

run_listing(8,0)
run_churn_listing.run_listing(7,[3,4,6,8],insert=True)
run_churn_listing.run_listing(chapter=7,listing=[1],version=[1,2,3,4,5,6],insert=True)

Then when I ran "run_listing(8,0)", the below files are created and error message came,
Running chapter 8 listing 1 prepare_data on schema socialnet7
Saving results to output_jupyter/socialnet7/socialnet7_dataset2_summarystats.csv
Saving results to output_jupyter/socialnet7/socialnet7_dataset2_scores.csv
Saving params to output_jupyter/socialnet7/socialnet7_dataset2_score_params.csv

ValueError Traceback (most recent call last)
/var/folders/b7/2m41z5cd2p9c5qrqt0xrmz7c0000gn/T/ipykernel_1835/1734770269.py in
2 # run_churn_listing.run_listing(7,[3,4,6,8],insert=True) #
3 # run_churn_listing.run_listing(chapter=7,listing=[1],version=[1,2,3,4,5,6],insert=True)
----> 4 run_listing(8,1)

~/Desktop/Life/DS Lab/Churn Modeling/fightchurn/run_churn_listing.py in run_listing(chapter, listing, version, schema, insert)
338 schema=schema,
339 insert=insert)
--> 340 run_churn_listing_from_args(args)
341
342

~/Desktop/Life/DS Lab/Churn Modeling/fightchurn/run_churn_listing.py in run_churn_listing_from_args(args)
353 if len(args.version)==0:
354 list_args.version=None
--> 355 run_listing_from_args(list_args)
356 else:
357 for v in args.version:

~/Desktop/Life/DS Lab/Churn Modeling/fightchurn/run_churn_listing.py in run_listing_from_args(args)
287 _sql_listing_from_params(listing_params,args.insert)
288 elif type=='py':
--> 289 _python_listing_from_params(listing_params)
290 else:
291 raise Exception('Unsupported type %s' % type)

~/Desktop/Life/DS Lab/Churn Modeling/fightchurn/run_churn_listing.py in _python_listing_from_params(param_dict)
157 name = param_dict.pop('name')
158
--> 159 python_listing(chapter, listing, name, param_dict)
160
161

~/Desktop/Life/DS Lab/Churn Modeling/fightchurn/run_churn_listing.py in python_listing(chapter, listing, name, param_dict)
189 if ex_fun is not None:
190 # Run the function, passing the parameters
--> 191 ex_fun(**example_params)
192 else:
193 print('Could not find function %s in module %s' % (name , full_name))

~/Desktop/Life/DS Lab/Churn Modeling/fightchurn/listings/chap8/listing_8_1_prepare_data.py in prepare_data(data_set_path, group_corr_thresh)
8 dataset_stats(data_set_path)
9 fat_tail_scores(data_set_path)
---> 10 find_metric_groups(data_set_path,group_corr_thresh)
11 apply_metric_groups(data_set_path)
12 ordered_correlation_matrix(data_set_path)

~/Desktop/Life/DS Lab/Churn Modeling/fightchurn/listings/chap6/listing_6_4_find_metric_groups.py in find_metric_groups(data_set_path, group_corr_thresh)
63 metric_columns = list(score_data.columns.values)
64
---> 65 labels = find_correlation_clusters(score_data.corr(),group_corr_thresh)
66 labeled_column_df, relabled_count = relabel_clusters(labels,metric_columns)
67 loadmat_df = make_load_matrix(labeled_column_df, metric_columns, relabled_count,group_corr_thresh)

~/Desktop/Life/DS Lab/Churn Modeling/fightchurn/listings/chap6/listing_6_4_find_metric_groups.py in find_correlation_clusters(corr, corr_thresh)
9 def find_correlation_clusters(corr,corr_thresh):
10 dissimilarity = 1.0 - corr
---> 11 hierarchy = linkage(squareform(dissimilarity), method='single')
12 diss_thresh = 1.0 - corr_thresh
13 labels = fcluster(hierarchy, diss_thresh, criterion='distance')

~/anaconda3/lib/python3.9/site-packages/scipy/spatial/distance.py in squareform(X, force, checks)
2213 raise ValueError('The matrix argument must be square.')
2214 if checks:
-> 2215 is_valid_dm(X, throw=True, name='X')
2216
2217 # One-side of the dimensions is set here.

~/anaconda3/lib/python3.9/site-packages/scipy/spatial/distance.py in is_valid_dm(D, tol, throw, name, warning)
2288 if not (D == D.T).all():
2289 if name:
-> 2290 raise ValueError(('Distance matrix '%s' must be '
2291 'symmetric.') % name)
2292 else:

ValueError: Distance matrix 'X' must be symmetric.

Just wondering if there is any dependency I missed.

Looking forward to hearing from you!

Best
Josh

Dear authors,

First of all, I would like to thank you for the amazing content that you put on the blog (https://fightchurnwithdata.com/churn-blog/), currently, I am going through the book and wanted to build a POC of churn prediction. Unfortunately during the installation of your package, I have encountered an error (while installing the "shap==0.39.0" package). Although I understand that the error has less to do with your package but with the sharp installation of v0.39.0 (and for that, I already wrote to the shap contributors), I would still like to ask if it is possible to update the requirements with other latest version shap==0.40.0 which is easily installable?
Thanks in advance!

The listing in chapter 4 to calculate churn periods don't work on Redshift and Snowflake as they don't support UNION, only UNION ALL. UNION ALL then runs into max_rows_recursion limits (only tried on Redshift) very quickly as it doesn't remove the duplicates from the previous run.

When trying to run listing 3.5 using run_churn_listing.py an error is returned, "No listing for chapter 3 in socialnet7_listings.json"

I can see that there is no list5 item in the chap3 section of socialnet7_listings.json.

Hey, I'm having an issue with the generation of data step.

Getting the errot ModuleNotFoundError: No module named 'fightchurn'

Hello, i follow this book and try to setup
every thing work ok but when i run churndb, it create table and schema, but maybe it not create primary key
because pgadmin table not have pm key, it lock and not allow me to import data
when i setup primary key, it raise error duplicate pmkey:
psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "event_pkey"
DETAIL: Key (account_id)=(1) already exists.
CONTEXT: COPY event, line 2
So i can't import data into it, what should i do now