Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pictogram #52

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
112 changes: 80 additions & 32 deletions src/engine.ts
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,46 @@ async function getWordSuggestions({
throw new Error("ERROR: Suggestion list is empty");
}

// A function to generate a prompt for generating images from Leonardo AI using GPT3.5-turbo-instruct and provided template and words
export async function generatePromptForImageGeneration({
word,
}: {
word: string;
}): Promise<string> {
const completionRequestParams = {
model: "gpt-3.5-turbo-instruct",
prompt:
`Create a detailed prompt to generate a pictogram for '${word}'.
First, determine if this is primarily an ACTION or OBJECT, then create a prompt following the appropriate template below.

For ACTIONS (verbs, activities):
- Show a figure actively performing the action
- Include clear motion indicators where appropriate
- Focus on the most recognizable moment of the action
- Use side view if it better shows the action
- Include minimal but necessary context elements

Style requirements:
- Bold black outlines
- Flat colors
- High contrast
- Centered composition
- White background
- Simple geometric shapes

Return only the prompt, no explanations. Keep it under 100 words.`,
temperature: 0,
max_tokens: 150,
};

const response = await globalConfiguration.openAIInstance.createCompletion(
completionRequestParams
);
const prompt = response.data?.choices[0]?.text;
if (!prompt) throw new Error("Error generating prompt for image generation");
return prompt;
}

Comment on lines +124 to +163
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job! Please create a separate PR with these changes—they look fantastic! Additionally, open a new PR in the Cboard-AI-Builder to implement this functionality so we can test it thoroughly

async function fetchPictogramsURLs({
words,
language,
Expand Down Expand Up @@ -160,7 +200,30 @@ async function checkWordAvailability(
symbolSet,
globalSymbolsSymbolSet,
});
return urls[0].pictogram.images.length > 0 && urls[0].pictogram.images[0].url !== "";
return (
urls[0].pictogram.images.length > 0 &&
urls[0].pictogram.images[0].url !== ""
);
}

async function processBatch<T>({
items,
batchSize,
processor,
}: {
items: T[];
batchSize: number;
processor: (batch: T[]) => Promise<any>;
}): Promise<any[]> {
const results: any[] = [];

for (let i = 0; i < items.length; i += batchSize) {
const batch = items.slice(i, i + batchSize);
const batchResults = await processor(batch);
results.push(...batchResults);
}

return results;
}

async function getSuggestions({
Expand All @@ -176,46 +239,31 @@ async function getSuggestions({
symbolSet?: SymbolSet;
globalSymbolsSymbolSet?: string;
}): Promise<Suggestion[]> {
const words: string[] = await getWordSuggestions({
const words = await getWordSuggestions({
prompt,
maxWords: maxSuggestions * 2,
language,
});

const validatedWords: string[] = [];
const unavailableWords: string[] = [];

for (const word of words) {
if (validatedWords.length >= maxSuggestions) {
break;
}

const isAvailable = await checkWordAvailability(
word,
const suggestions = await processBatch({
items: words,
batchSize: 5,
processor: (batch) => fetchPictogramsURLs({
words: batch,
language,
symbolSet,
globalSymbolsSymbolSet
);

if (isAvailable) {
validatedWords.push(word);
} else {
unavailableWords.push(word);
}
}
//In case the number of validated words is less than the maxSuggestions, we add unavailable words to reach the maxSuggestions
while (validatedWords.length < maxSuggestions) {
validatedWords.push(unavailableWords.pop() || "");
}

return await fetchPictogramsURLs({
words: validatedWords,
language,
symbolSet,
globalSymbolsSymbolSet,
globalSymbolsSymbolSet,
}),
});
}

const validSuggestions = suggestions.filter(
suggestion =>
suggestion.pictogram.images.length > 0 &&
suggestion.pictogram.images[0].url !== ""
);

return validSuggestions.slice(0, maxSuggestions);
Comment on lines +259 to +265
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if the validSuggestions array is not large enough to reach the maxSuggestions limit? In that case, we should consider filling the remaining slots with fallback or alternative suggestions, ensuring the final results still meet the expected size.

}

async function isContentSafe(textPrompt: string): Promise<boolean> {
try {
Expand Down