VertexAI Gemini Safety Settings: How to determine offending text?

Question

Gemini SAFETY_SETTINGS used:

• I used to use BLOCK_NONE but a week or so ago, the API starting blocking this unless you pay $40k/mo. in fees (at least that is what the URL in the error code stated).
• Now, BLOCK_ONLY_HIGH is causing issues with certain non-invasive topics.

SAFETY_SETTINGS = [
        SafetySetting(
            category=HarmCategory.HARM_CATEGORY_HARASSMENT,
            threshold=HarmBlockThreshold.BLOCK_ONLY_HIGH
        ),
        SafetySetting(
            category=HarmCategory.HARM_CATEGORY_HATE_SPEECH,
            threshold=HarmBlockThreshold.BLOCK_ONLY_HIGH
        ),
        SafetySetting(
            category=HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT,
            threshold=HarmBlockThreshold.BLOCK_ONLY_HIGH
        ),
        SafetySetting(
            category=HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT,
            threshold=HarmBlockThreshold.BLOCK_ONLY_HIGH
        )
    ]

Error:

{
    "Candidate": {
        "finish_reason": "RECITATION",
        "safety_ratings": [
            {
                "category": "HARM_CATEGORY_HATE_SPEECH",
                "probability": "NEGLIGIBLE",
                "probability_score": 0.40625,
                "severity": "HARM_SEVERITY_NEGLIGIBLE",
                "severity_score": 0.091308594
            },
            {
                "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
                "probability": "NEGLIGIBLE",
                "probability_score": 0.22949219,
                "severity": "HARM_SEVERITY_LOW",
                "severity_score": 0.25585938
            },
            {
                "category": "HARM_CATEGORY_HARASSMENT",
                "probability": "NEGLIGIBLE",
                "probability_score": 0.4453125,
                "severity": "HARM_SEVERITY_LOW",
                "severity_score": 0.37695312
            },
            {
                "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
                "probability": "NEGLIGIBLE",
                "probability_score": 0.12597656,
                "severity": "HARM_SEVERITY_NEGLIGIBLE",
                "severity_score": 0.11767578
            }
        ],
        "citation_metadata": {
            "citations": [
                {
                    "start_index": 3396,
                    "end_index": 10842
                }
            ]
        },
        "avg_logprobs": "NaN"
    },
    "Response": {
        "candidates": [
            {
                "finish_reason": "RECITATION",
                "safety_ratings": [
                    {
                        "category": "HARM_CATEGORY_HATE_SPEECH",
                        "probability": "NEGLIGIBLE",
                        "probability_score": 0.40625,
                        "severity": "HARM_SEVERITY_NEGLIGIBLE",
                        "severity_score": 0.091308594
                    },
                    {
                        "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
                        "probability": "NEGLIGIBLE",
                        "probability_score": 0.22949219,
                        "severity": "HARM_SEVERITY_LOW",
                        "severity_score": 0.25585938
                    },
                    {
                        "category": "HARM_CATEGORY_HARASSMENT",
                        "probability": "NEGLIGIBLE",
                        "probability_score": 0.4453125,
                        "severity": "HARM_SEVERITY_LOW",
                        "severity_score": 0.37695312
                    },
                    {
                        "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
                        "probability": "NEGLIGIBLE",
                        "probability_score": 0.12597656,
                        "severity": "HARM_SEVERITY_NEGLIGIBLE",
                        "severity_score": 0.11767578
                    }
                ],
                "citation_metadata": {
                    "citations": [
                        {
                            "start_index": 3396,
                            "end_index": 10842
                        }
                    ]
                },
                "avg_logprobs": "NaN"
            }
        ],
        "usage_metadata": {
            "prompt_token_count": 1298239,
            "total_token_count": 1298239,
            "cached_content_token_count": 1297553
        }
    }
}

As you can see, the response in no where near HIGH...Whats going on here?

Question is: Is the start / end index a character count or a token count?

• I can't find this info anywhere on the Gemini docs.

---

EDIT 1:

@sourav, The settings shown in the link you provided look like the ones I defined.

Answer 1

You can try with the below format :

HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_ONLY_HIGH,
HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_ONLY_HIGH,
HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_ONLY_HIGH,
HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_ONLY_HIGH

The issue seems to be with the format of your SAFETY_SETTINGS dictionary. In your code, you're using a list of dictionaries, each containing a category and threshold key. However, the SAFETY_SETTINGS dictionary should be a single dictionary where the keys are the harm categories and the values are the thresholds.

For gcp gemini documentation you can refer this one Gemini API SDKs and How to configure safety filters