Comments (4)
Thanks for sharing the detailed instructions @sharadregoti!
I tried this rail spec which worked for me -- can you give it a shot:
<rail version="0.1">
<output>
<integer name="epf" description="Employee Provident Fund Amount (EPF) per annum" />
<integer name="gratuity" description="Gratuity per annum" />
<integer name="medialInsurance" description="Medical Insurance per annum" />
<integer name="termInsurance" description="Term Insurance per annum" />
<integer name="ctc" description="Cost To Company per annum" />
<object name="miscellaneous" description="Cost To Company per annum">
</object>
</output>
<prompt>
I have shared sample data of offer letter which has a CTC amount and it's breakdown in it
{{table}}
@complete_json_suffix_v2
</prompt>
</rail>
The main change is in the spec, I changed the output type from string
to integer
for all dictionary values. This corrected the parser error, which led to validation proceeding as expected.
from guardrails.
I had tried changing string
to integer
. But bad luck Same result.
There is this jsondecoder error, can this be an issue
-- message: {"'output'": '\'{\\n "epf": 21,600,\\n "gratuity": 35,760,\\n "medialInsurance": 3,060,\\n "termInsurance": 3,672,\\n "ctc": 1,583,548,\\n "miscellaneous": {\\n "HRA": 371,748,\\n "specialAllowance": 305,148,\\n "internet": 30,000,\\n "technicalBooks": 10,000,\\n "giftVoucher": 5,004,\\n "pvAmount": 53,112,\\n "ctcMedicalPrem": 948\\n }\\n}\'', "'output_as_dict'": 'None', "'error'": "JSONDecodeError('Expecting property name enclosed in double quotes: line 2 column 15 (char 16)')", "'timestamp'": '1684915582.1477263', "'task_uuid'": "'28d0be01-469f-470a-ba1b-d4fead86aa14'", "'task_level'": '[2, 2, 3, 2]', "'message_type'": "'info'"}
from guardrails.
@sharadregoti can you try this rail spec?
<rail version="0.1">
<output>
<integer name="epf" description="Employee Provident Fund Amount (EPF) per annum" />
<integer name="gratuity" description="Gratuity per annum" />
<integer name="medialInsurance" description="Medical Insurance per annum" />
<integer name="termInsurance" description="Term Insurance per annum" />
<integer name="ctc" description="Cost To Company per annum" />
<object name="miscellaneous" description="Cost To Company per annum">
</object>
</output>
<instructions>
You are a helpful assistant only capable of communicating with valid JSON, and no other text.
@json_suffix_prompt_examples
</instructions>
<prompt>
I have shared sample data of offer letter which has a CTC amount and it's breakdown in it.
{{table}}
If extracting any integer value, make sure to extract it as a number and not as a string.
This means that if the value is 1,00,000, then it should be extracted as 100000 and not as 1,00,000.
If you are unable to extract any value, use `null`.
@xml_prefix_prompt
{output_schema}
</prompt>
</rail>
I did some prompt engineering to make it work with your table. The issue that was happening was that the source table had numbers with commas, which was messing with the json decoding.
Also, when you use this, I recommend using this with temperature 0.0.
I tested this out with gpt-3, gpt-3.5 and gpt-4, and it worked across all 3.
# GPT-3
raw_llm_output, validated_output = guard(
openai.Completion.create,
prompt_params={"table": table},
engine="text-davinci-003",
max_tokens=1024,
temperature=0.0,
)
# GPT-3.5
raw_llm_output, validated_output = guard(
openai.ChatCompletion.create,
prompt_params={"table": table},
model="text-davinci-003",
max_tokens=1024,
temperature=0.0,
)
from guardrails.
Closed due to inactivity. Feel free to reopen.
from guardrails.
Related Issues (20)
- [bug] litellm support erroring for azure openai falling through to the openai callable HOT 1
- [bug]ImportError: cannot import name 'ExpiredSignatureError' from 'jwt' HOT 1
- [documentation] update RAIL docs for hub validators
- [bug] Error after installing SensitiveTopics, error: ModuleNotFoundError: No module named 'guardrails.hub.tryolabs' HOT 3
- [bug] Error: zsh: bus error guardrails start --config config.py HOT 1
- Update hub validators to support remote inference
- [bug] email address [email protected] not receiving emails: "Delivery Status Notification (Failure)" HOT 1
- [docs] request: Add doc examples of how to call the new Guardrail Server endpoints. HOT 1
- [bug] Trouble calling guard without using openai or azure openai APIs HOT 6
- [docs][Bug] HOT 1
- How to use gemini model insted of openai HOT 2
- [feat] Functionality for testing a Guard configured to run on prompt in isolation HOT 1
- [feat] More readable validation failures HOT 2
- [docs] Missing docs for in-code hub installs
- [bug] Toxic language validator returning 500 HOT 10
- [feat] Support punkt_tab instead of punkt HOT 1
- [bug] Guards that dont exist in settings.use_remote = True dont error and openAI api error for guard not found not helpful HOT 2
- [bug] ToxicLanguage validator's "fix" behaviour breaks "fix-reask"
- [bug] GuardRunnable may not be reporting tokens appropriately if reasks are allowed
- [bug] Verify Filter, Reask, and Refrain ser/de
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from guardrails.