Python: Content Filtering not triggering as expected

[Justrebl]

Hello,

I've been playing with the Agent Framework, and built the simplest ChatAgent just to test the format of content filtering in the agent.run() result.
Using a demo example from the guardrail testing in Azure AI Foundry portal, i can see the content filtering trigger as expected :

Running the exact same prompt to my basic agent, i actually retrieve an information, no content-filtering triggered, same AI Foundry Project, model and content-filtering configurations :

Here's the code behind :

@app.route('/ask', methods=['POST'])
async def ask():
    data = request.get_json()
    try:
        model = Query(**data)
        async with (
            AzureCliCredential() as credential,
            ChatAgent(
                chat_client=AzureAIAgentClient(async_credential=credential),
                instructions="You are a helpful assistant."
            ) as agent,
        ):
            await agent.chat_client.setup_azure_ai_observability(enable_sensitive_data=True)
            result = await agent.run(model.query)
            return jsonify({
                'query': model.query,
                'sessionId': model.sessionId,
                'answer': result.text
            })

Am I missing something ?
Thanks !

[markwallace-microsoft]

@dmytrostruk Julien has offered to help with reproducing this if you need assistance.

[dmytrostruk]

Hi @Justrebl ! Sorry for delayed response!

I can reproduce this case. Based on my testing, I believe the content filtering works differently on portal compared to programmatic access. It looks like on portal it's applied by default, while in the code you have to specify it explicitly. I checked azure.ai.agents package (that is used by AzureAIAgentClient from your example) and I couldn't find any parameter that would allow to specify the policy.

However, I also checked azure.ai.projects package (which is used by new AzureAIClient Responses-based API) and there is a way how to specify the policy and trigger the content filtering there (by using RaiConfig). I verified and it works as expected.

I provided an example in this PR: #3265