Currently, the function makes sequential requests to retrieve each pictogram, filters them to check for availability, and then re-fetches to create a new list of validated suggestions. This results in redundant API calls and can impact performance.

Suggested Improvements:

Create the Suggestions Array While Fetching: Build the Suggestions array during the initial availability checks, avoiding unnecessary re-fetching of images. This will improve both performance and API efficiency.

Batch Fetch Requests with Limited Concurrency: Instead of fetching each image individually, you can use fetchPictogramsURLs to batch the requests for multiple words at once. By limiting the number of concurrent requests (e.g., fetching 5 words at a time), you can significantly reduce the number of sequential API calls, improving efficiency and reducing API load.

These adjustments should lead to better performance and make the function more efficient.

Make sense. Will implement!

Great job! Please create a separate PR with these changes—they look fantastic! Additionally, open a new PR in the Cboard-AI-Builder to implement this functionality so we can test it thoroughly

Please remove this function, as it is no longer in use

Awesome work! Let's try to define the types for the function:

async function processBatch<T,F>({
  items,
  batchSize,
  processor,
}: {
  items: T[];
  batchSize: number;
  processor: (batch: T[]) => Promise<F[]>;
}): Promise<F[]> {
  const results: F[] = [];
  
  for (let i = 0; i < items.length; i += batchSize) {
    const batch = items.slice(i, i + batchSize);
    const batchResults = await processor(batch);
    results.push(...batchResults);
  }
  
  return results;
}

And use it like

  const suggestions = await processBatch<string,Suggestion>({
    items: words,
    batchSize: 5,
    processor: (batch) => fetchPictogramsURLs({
      words: batch,
      language,
      symbolSet,
      globalSymbolsSymbolSet,
    }),
  });

Alternatively, you can define the exact types instead of using generics,

Another point to consider: Currently, we are fetching all the Arasaac images for the words array. Instead, we could validate each image as it is returned, populate the results array dynamically, and stop fetching once the array is fully populated. This approach would help us avoid unnecessary API requests and improve performance.

What happens if the validSuggestions array is not large enough to reach the maxSuggestions limit? In that case, we should consider filling the remaining slots with fallback or alternative suggestions, ensuring the final results still meet the expected size.