Proposal to add prompt engineering tools

[zhimin-z]

How about tools such as https://x.ai/blog/prompt-ide, https://github.com/stanfordnlp/dspy, https://github.com/microsoft/prompty, https://github.com/bigscience-workshop/promptsource, and https://github.com/MadcowD/ell? @axsaucedo

[axsaucedo]

I'm not yet clear how on prompt engineering in this list, I do see huge value in this field but even looking at the tools proposed in the PR, I fail to see how these are tools for production machine learning as opposed to application-level tooling to leverage / interact with LLMs (e.g. https://github.com/promptslab/Awesome-Prompt-Engineering). I would see something along the lines of New Section on GenAI via #339 as more relevant to be explored in favour of something specifiaclly around prompt engineering tooling.

[zhimin-z]

I'm not yet clear how on prompt engineering in this list, I do see huge value in this field but even looking at the tools proposed in the PR, I fail to see how these are tools for production machine learning as opposed to application-level tooling to leverage / interact with LLMs (e.g. https://github.com/promptslab/Awesome-Prompt-Engineering). I would see something along the lines of New Section on GenAI via #339 as more relevant to be explored in favour of something specifiaclly around prompt engineering tooling.

Good catch, @axsaucedo . The logic behind prompt engineering tools is that they could foster the deployment of LLM/StableDiffusion in industry production. For example, the artists could try with various painting style using the prompt engineering tools and this not only save their time and efforts but also bring the underlying power of StableDiffusion models to real-world applications. So does LLM-related prompt tools as listed in the PR.

[axsaucedo]

the artists could try with various painting style using the prompt engineering tools and this not only save their time and efforts but also bring the underlying power of StableDiffusion models to real-world application

Just to clarify I am not challenging the usefulness of production usecases of promp engineering, but I don't see it as a section for production ML frameworks at this stage, especially looking at the PR.

The tools proposed in the PR seem to be: 1) click-prompt is only a UI for running models - not relevant for this list; 2) promptify is just an SDK to interact with OpenAI API - not relevant for this list; 3) Promptsource is an sdk to save templates for prompts - closest section would be data mgmt but I wouldnt' consider this relevant for this list at this stage.

There is quite a lot of hype going around with these tools so I don't want to jump on the bandwagon and add sections just because a lot of talk around it or a lot of potential - certainly keen to revisit eventually, but at this stage based on these examples I don't see a prompt engineering section being added at this stage, instead I woudl suggest we continue the discussion on how a GenAI or LLMs section that focuses on productionisation tools for these type of AI - however as per the discussion it does seem like there's ambiguity there but keen to continue exploring.

[zhimin-z]

How about tools such as https://x.ai/blog/prompt-ide, https://github.com/stanfordnlp/dspy, https://github.com/microsoft/prompty, https://github.com/bigscience-workshop/promptsource, and https://github.com/MadcowD/ell? @axsaucedo