enhanced model for tamil speech to text

[jpambattu]

Which Deepgram product feature are you using?

Deepgram API - STT

Details

I hadn't spoken anything but still it keeps on transcripting. I checked via code too (python sdk), same issue. I even muted all of my input devices; still no change.

Only Tamil has this issue, I tried en for same model, no issues
Screenshot is api playground and txt file is the code output. Check the confidence of the transcripts!!!
error.txt

If you are making a request to the Deepgram API, what is the full Deepgram URL you are making a request to?

No response

If you are making a request to the Deepgram API and have a request ID, please paste it below:

No response

If possible, please attach your code or paste it into the text box.

options: LiveOptions = LiveOptions(
            model="enhanced",
            punctuate=True,
            language="ta",
            encoding="linear16",
            channels=1,
            sample_rate=16000,
            # To get UtteranceEnd, the following must be set:
            # interim_results=True,
            # utterance_end_ms="1000",
            # vad_events=True,
)

If possible, please attach an example audio file to reproduce the issue.

No response

[team-deepgram]

Thanks for asking your question about Deepgram! If you didn't already include it in your post, please be sure to add as much detail as possible so we can assist you efficiently, such as:

• The request_id if you have a question about your requests or transcription responses.
• The features you used or the full api.deepgram.com URL you sent your request to, including parameters.
• Any code snippets you can share.

[jpvajda]

@jpambattu I'm wondering if you could try something out.

It seems you've tried this our Console Live Streaming missions and also tested via the Python SDK.

Though in the Console Mission (the screen shot) you've provided that green microphone button should disable the mic on you device. I just tested this mission with Tamil and didn't have an issue on my end using FireFox.

Given you are also seeing this in an SDK example I'm wondering if it's a device or browser specific issue for you, and not having anything to do with Deepgram the service.

Suggestion
If you can look at this Live Python Starter App and try to running it locally, try setting the model to Enhanced with a language option of Tamil Let me know if you see the same issue. The code is here:

If you do see a similar issue -double check your microphone settings in your browser sometimes that can be an issue.

[jpambattu]

Thanks for the quick reply.

I checked with the given Starter App code and in another browser, also with another input device. Issue persists!!!

Only for Tamil is the issue; I double checked everywhere with English but no issues at all.

[jpvajda]

@jpambattu can you give me a sense of what Device + OS + Browser you are seeing this behavior in? I'm pretty confident this has nothing to do with the Deepgram API or the Model itself, based on my testing.

[jpambattu]

Device and OS info.

Google chrome and Microsoft Edge browsers.

Python SDK and Starter App is tested in Python 3.11.7

[jpvajda]

@jpambattu I can't reproduce this and this isn't an issue anyone else has faced with our Tamil Model at this point so I'd have to say this must be a local issue for you and I don't have a lot to offer. Sorry I can't get you to a resolution, but play around with some of your OS or browser settings or perhaps try the FireFox Browser.

[jpambattu]

Thanks for the support. I am bewildered too.

So instead of checking in my laptop now I tested it on my Android 14 device.
Three different browsers I used. Google Chrome, 1DM+ and Firefox. Issue persists.
Below are the screenshots respectively.

So browser issues let it be, but why the same issue is happening when I am running the code. If it is a browser issue, then it should not happen when I run the code, now it is not a local issue as it is reproduced in my Android device.

I think even when the mic is muted your api will still get the data, and if you plot it you can still see the sine wave. My hypothesis is api/model is processing that data and generates the output; that's why, if you observe the output, same text is repeatedly generated!!!

Please let me know if you need more data on this.

[jpvajda]

@jpambattu Thanks for being patient and working through this with us, I pulled in @DamienDeepgram who was able to pinpoint the issue and we found out that the model seems to be hallucination on silence and returning text which makes it seem like the webSocket isn't closing and the mic is staying on.

So what you saw is definitely an issue we are going to look into at the model level.

[jpvajda]

@jpambattu . Given this issue is persisting at the model level I don't believe the issue is with our code with the SDK itself or in the Console Mission you saw this behavior in.

[jpambattu]

Okay. So other regional language models also might have the same hallucination. I was looking for Tamil speech to text service. How can I proceed then ?

[jpvajda]

@jpambattu I'd suggest you let us look at the issue in the model itself and see what needs to be done to correct the hallucination problem, I don't have an ETA of when it might be resolved but I've let our Engineering team know about the problem.

[jpambattu]

@jpvajda thanks for the support. Sure. Happy to help.

[venkat-iscalepro]

It looks like this issue is still occurring. I tried from the deepgram console. Could you let me know the ETA when it would be fixed.