CSV Agent - OpenAI Context limit

[infa-bhbalaji]

I am trying to load a large CSV with create_csv_agent function.
I get the error " This model's maximum context length is 4097 tokens, however you requested 6595 tokens" when I do agent.run() for some commands. I tried looking for ways to use document splitter here. But it is not clear from the documentation about how to integrate a split document with agents directly.

[Adorp94]

I'm running through the same issue. What's odd for me is that I was able to run it some weeks ago without this issue. Have found the way to solve it? Or have you applied an strategy that worked (i.e. map reduce, refining, etc).

Thanks. I will be working on how to solve it. If I come with an asnwer I'll respond here.

[TMoneyBidness]

I have the same issue. It's infuriating. Sometimes it works, others it doesn't, I have a CSV with 100 rows.