Comments (9)
I've noticed exceptions like KeyError being thrown in a few places if the rail spec is invalid. Maybe we should test whether the rail spec is valid before running it through @ShreyaR?
from guardrails.
This is my rail spec:
<rail version="0.1">
<output>
<object name="location_info">
<string name="name" description="Location's name" />
<choice
name="category"
description = "The function of the location. This can be a 'restaurant', 'shop', 'parking_spot' or 'other'."
format="valid-choices: {['restaurant', 'shop', 'parking_spot','other']}"
on-fail="filter">
<case name="restaurant">
<object name="restuarant_info">
<string
name="type_food"
description = "Which type of food do they serve here. Fit the answer in one of the following options: 'African', 'Asian', 'Dessert_Bakery', 'Friture', 'Mediterranean', 'Middle_Eastern', 'Western'."
format="valid-choices: {['African', 'Asian', 'Dessert_Bakery', 'Friture', 'Mediterranean', 'Middle_Eastern', 'Western']}"
/>
<bool
name="vegan"
description="if the restuarant is vegan or not"
/>
<bool
name="vegatarian"
description="if the restuarant is vegatarian or not"
/>
</object>
</case>
<case name="shop">
<object name="shop_info"></object>
</case>
<case name="parking_spot">
<object name="parking_spot_info"></object>
</case>
<case name="other">
<object name="other_info"></object>
</case>
</choice>
<email name="email" description="the email address of the location" on-fail="fix"/>
<object name="address">
<string name="street" description="the streetname of the location" on-fail="filter"/>
<integer name="number" description="the streetnumber of the location" on-fail="filter"/>
<integer name="postal_code" description="the postal code of the location" on-fail="filter"/>
<string name="city" description="the city name of the location" on-fail="filter"/>
<string name="country" description="the country name of the location" on-fail="filter"/>
</object>
<url name="website" description="an url leading to the website of the location" />
<integer name="entrance_door_width" description="the width of the door in centimeters"/>
<object name="steps">
<integer
name="amount_of_steps"
description="the amount of steps at the entrance" />
<integer
name="total_height_of_the_steps"
description="the total height of the steps in centimeters"
if="amount_of_steps!=0 and amount_of_steps!='None' and amount_of_steps!=None"/>
<bool
name="access_ramp"
description="Is there access to a ramp to go over the steps at the entrance."
if="amount_of_steps!=0 and amount_of_steps!='None' and amount_of_steps!=None"/>
</object>
<bool
name="turning_point"
description="..."/>
<object name="toilet">
<bool
name="toilet_present"
description="Is there a toilet present at the location"/>
<string
name="situation_of_toilet"
description="Is the toilet accesible for wheelschair users or is it a standard toilet"
if="toilet_present==True"
format="valid-choices: {['Accesible', 'Standard']}"
on-fail="filter"/>
<choice
name="location_of_the_toilet"
if="toilet_present==True"
description="How the toilet can be reached, with stairs, a lift or is it on the ground floor."
format="valid-choices: {['lift', 'ground_floor','stairs']}"
on-fail="filter">
<case name="lift">
<object name="lift_info">
<integer
name="lift_door_width"
description = "The width of the door of the elevator in centimeters"/>
<integer
name="lift_width"
description="The width of the elevator on the inside" />
<integer
name="lift_depth"
description="The depth of the elevator on the inside" />
</object>
</case>
<case name="ground_floor">
<object name="ground_floor_info"></object>
</case>
<case name="stairs">
<object name="stairs_info"></object>
</case>
</choice>
<integer
name="width_narrowest_point_from_entrance_to_toilet"
if="toilet_present==True"
description="the width of the narrowest point from the entrance to the toilet in centimeters"
/>
<integer
name="toilet_door_width"
if="toilet_present==True"
description="the width of the door of the toilet in centimeters"
/>
<integer
name="number_of_handrails"
if="toilet_present==True"
description="amount of handrails available in the toilet"
/>
<integer
name="free_space_infront_toilet"
if="toilet_present==True"
description="amount of space in centimeters in front of the toilet"
/>
<integer
name="free_space_next_toilet"
if="toilet_present==True"
description="amount of space in centimeters next to the toilet"
/>
<bool
name="pay_toilet"
if="toilet_present==True"
description="Do you have to pay to use the toilet?"
on-fail="filter"/>
<bool
name="baby_changing_table"
if="toilet_present==True"
description="if there is a baby changing table present in the toilet"/>
<bool
name="adult_changing_table"
if="toilet_present==True"
description="if there is a adult changing table present in the toilet"/>
</object>
</object>
</output>
<instructions>
You are a helpful assistant only capable of communicating with one valid JSON, and no other text.
@json_suffix_prompt_v2
</instructions>
<prompt>
Given the following information about a location, please construct a JSON that follows the correct schema.
If the answer doesn't exist in the user input, enter Null. This applies to integers, booleans and strings.
If you want to use a boolean: use 'True' or 'False' as strings, but only if the user mentioned this variable. Otherwise you enter Null.
Please extract a JSON that contains all the location's information. Be sure to go over every variable in the XML format.
The input from the user is: "{{user_input}}".
@json_suffix_prompt_v2
{output_schema}
The following variables are already given:
{{memory}}
</prompt>
</rail>
Do you have advise on what I did wrong to cause the error?
from guardrails.
Ah, you mean with a valid rail spec, if the document does not contain any information regarding the choice, it throws the KeyError?
Thanks for providing the rail spec, could you also provide the full prompt you used (including the user input)?
from guardrails.
The memory was empty and the user input was: "The name of the restaurant is Jam. There are 4 steps in front of the door"
from guardrails.
Because I didn't mention anything about a toilet. It set all the corresponding variable to None, which led to the error in the choice variable 'location_of_the_toilet'.
from guardrails.
I've prepared a PR that fixes this.
By the way, if="toilet_present==True"
isn't valid guardrails syntax. The only time you see if
in your schema is if you use choice, and it's compiled to that form after specification. Has it worked well for you?
from guardrails.
Thank you for the fix! And thanks for the tip! The LLM seems to be following the if condition so far.
from guardrails.
Cool! Consider adding required="false"
to the fields with the if condition, otherwise it might break under #185.
from guardrails.
Hey, Choice works differently now (as a discriminated union), I think this bug doesn't exist anymore. If you encounter any problems, please open a new issue :)
from guardrails.
Related Issues (20)
- Add better_profanity as a validator which takes care of word dividers HOT 4
- Dead link to docs page HOT 2
- [bug] Importing guardrails raises a deprecation warning for old validators HOT 1
- [bug] JSON validation converts null values to True:bool HOT 1
- No capability to disable telemetry via env var [bug] HOT 2
- [bug] Message: guardrails-cli[6204] , ERROR Failed to inspect HOT 8
- Guardrails Langchain integration with streamable AgentExecutor HOT 7
- Automatically open `hub.guardrailsai.com/tokens` url when running guardrails configure HOT 1
- [feat] Support for image inputs in prompts HOT 1
- [bug] Endpoint is reachable Validator "Zero-click" Vulnerability HOT 5
- [bug]: Guardrails hub token not getting generated HOT 7
- Import Error - Cannot import name 'SensitiveTopic' from 'guardrails.hub' HOT 5
- How to Deploy Guardrails with Hub via Docker image HOT 4
- [bug] detect-prompt-injetion validator fails to install from hub HOT 2
- [feat] Support defaults
- [feat] add a `quiet` option to `guardrails hub install`
- [feat] Update validator imports in notebooks to hub
- [bug] llm_api function triggering twice HOT 6
- [bug] OpenAI v1.x AsyncClient Does Not Work in Guardrails as Expected
- [bug] Guardrails prohibits OpenAI from utilizing env-var based API key HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from guardrails.