Model features: native async

[raspawar]

Native Async

Validated current implementation and added test cases.

llm = ChatNVIDIA()
message = HumanMessage(content="Hello")
response = await llm.agenerate([[message]])

cc: @sumitkbh

[mattf]

@efriis we have async implementations for generation functions. what is the criteria for "native async" (see https://python.langchain.com/docs/integrations/chat/nvidia_ai_endpoints/#model-features)?

[mattf]

per discussion w/ Erick, we need to implement _agenerate, which will entail a move from requests to httpx

[mattf]

looking good. please add a test that confirms httpx requests are interleaved.

[mattf]

anthropic?

[mattf]

why make this optional?

[mattf]

should this be commented or not?

[mattf]

this will pass even if agenerate() is implemented without truly async communication w/ the server.

add a unit text that check that async generation requests are interleaved. for inspiration...

    async def afetch_data(url: str) -> str:
        async with httpx.AsyncClient() as client:
            return (await client.get(url)).text
    async def amock_response(request):
        await asyncio.sleep(1)
        return httpx.Response(200, text="Hello world!")

    start_time = time.time()

    httpx_mock.add_callback(amock_response, is_reusable=True)
    task1, task2 = afetch_data("http://example.com"), afetch_data("http://example.com")
    _, _ = await asyncio.gather(task1, task2)

    assert (time.time() - start_time) < 2, "Tasks did not run concurrently"