Error in Autocrew 2.0.7: Role Component Missing in CSV Data about autocrew HOT 10 CLOSED

Thanks for logging this. I've just released version 2.1.0, please give it a try

from autocrew.

Will do for sure. Hopefully can test it out this evening.

from autocrew.

If the issue is resolved in version 2.1.0 could you please close this issue?

from autocrew.

Please stress-test it and try to break it ;)

from autocrew.

Unfortunately I still seem to have the same issue. See log here: autocrew.log

from autocrew.

Hmm, that last one appeared to be an issue with the "scripts" dir not in existence though even after creating said dir I get another error: autocrew.log
EDIT: The files do appear to be created though at least the first two are incomplete. Attached is the output from running autocrew.py once: scripts.zip

from autocrew.

Thanks very much for this info.
In your 2nd autocrew.log file, the issue was that the script wasn't creating the scripts directory when wasn't present.
I've released an update moments which should fix this.

To upgrade, please backup everything in your current autocrew directory moving it to a completely different folder, then do the following:

cd ~
git clone https://github.com/yanniedog/autocrew

This will upgrade you to version 2.1.2.

You were actually using version 2.0.7, which is actually very different to the current version

from autocrew.

With regards to your first and third autocrew.log files, it's clear that in both those cases, the problem is that your LLM is unfortunately spitting unpredictable garbage, which doesn't fit the standard CSV format. The propensity for LLMs to do this is usually related to what the temperature setting for your LLM has been set to.

eg:

2024-01-28 08:22:45,314 - DEBUG - (function: get_agent_data, line: 254) - Raw LLM output (Ollama):
 "role","goal","backstory","assigned_task","allow_delegation"

"Editor1","\"Correctly punctuate & format\",""Works meticulously, has excellent attention to detail. Has experience in editing and formatting documents.''.",""Transcript editing & formatting.,\"Youtube transcript.,\"False.",

"Proofreader2","\"Correctly punctuate & format\",""Is a stickler for correct grammar and punctuation, always double-checks work for errors.''.",""Reviewing edited transcripts.,\"YouTube transcript proofreading.,\"False.",

"TeamLead3","\"Correctly punctuate & format team output\",""Effective communicator, experienced manager. Will ensure high quality of the completed dataset. ''.",""Coordinating editing & proofreading tasks.,\"Entire team's work on Youtube transcript.,\"True.",

"Specialist4","\"Correctly punctuate & format specialized sections\",""Has expertise in specific technical or academic terminology, will ensure that these sections are accurately formatted. ''.",""Identifying and formatting specialist sections.,\"Youtube transcript's technical or academic sections.,\"False."

and

2024-01-28 18:52:34,988 - DEBUG - (core.py,get_agent_data,185) - Raw LLM output (Ollama):
 "role","goal","backstory","assigned_task","allow_delegation"

"Agent1","Data Collector","Experienced in web scraping and data extraction. Has a knack for finding relevant information from various sources.","Fetch YouTube video URLs","True"
,"Agent2""Scraper","Skilled in using advanced web scraping tools to extract data from complex websites.","Assigned to fetch YouTube video URLs using specialized tools.","", "False"
, "Agent3" , "Video Downloader" , "Expert in downloading videos from various sources including YouTube.", "Download and format fetched YouTube videos.", "" , "True"
, "Agent4" , "Quality Assurance" , "Details-oriented with strong attention to detail. Responsible for ensuring the accuracy of data and formatting.", "Review and correct any errors or inconsistencies in downloaded and formatted YouTube videos.", "" , "False"
, "Agent5" , "Team Leader" , "Strong organizational skills and effective communicator. Oversees the entire process from URL collection to video download and formatting.", "", "" , "False"

The temperature setting required for each model will be slightly different, and configuring it from Autocrew's side is quite difficult.

On the other hand, I have had quite good, reliable results by using the standard openhermes model for ollama, and gpt-3.5-turbo for the openai api.

A lot of these kinds of issues were sorted out between 2.0.7 and 2.1.0, but it's difficult to process nonsensical output originating from the LLM. As they say, 'garbage in, garbage out'.

I'll keep trying to tweak the script to better respond to these kinds of LLM outputs. Will do my best.

(also, part of the reason I created the -m* command line parameter was so that, if the LLM is misbehaving, users could keep generating as many scripts as required until the LLM decides to spit out a properly structured CSV)

from autocrew.

Was just able to get mistral to spit out one without the issue!

Perhaps my previous prompt busted it or the roll of the dice was not in my favor. Anyway, I'll continue to monitor but will close this out for now as it sounds like its sort of out of your hands (& in the model's). Thanks for the help & advice!

from autocrew.

No dramas, will keep tweaking it. For script generation, definately try using openhermes. In the config.ini file you can make openhermes generate the scripts, and mixtral do the crewai functions within the scripts. For instance.

from autocrew.