Neural network GPT Prompt Engineer

GPT Prompt Engineer is an artificial intelligence tool designed to simplify the process of creating effective prompts for artificial intelligence systems. Leveraging the power of the GPT-4, GPT-3.5-Turbo, and Claude 3 Opus models, the service simplifies the complex task of tooltip creation by allowing users to generate, test, and rank multiple tooltips to find the most effective ones for their specific use cases.

Key Features

gpt-prompt-engineer tool offers a number of features that make it an invaluable asset for anyone looking to optimize AI performance:

1. Hint generation : Using advanced AI models, the tool can generate various possible hints based on the provided use case and test cases, providing hints customized to a specific task.

2. Hint testing : The system tests each hint on all test cases, comparing their performance and ranking them using the ELO rating system. This allows users to easily identify the most effective tips.

3. ELO Rating System : Each clue starts with an ELO rating of 1200. As they compete with each other to generate test case answers, their ELO ratings change based on their performance, providing a clear indication of the best performing clues.

4. Classification Version : The tool includes a classification version designed to perform classification tasks, scoring the correctness of a test case by comparing it to the expected output ('true' or 'false') and providing a score table for each hint.

5. Claude 3 version : The claude-prompt-engineer notebook is designed to work with Anthropic's Claude 3 Opus model, automatically generating test cases and allowing multiple input variables, making it even more powerful and flexible.

6. Logging of Weights & Biases : Additional logging in Weights & Biases of your configurations such as temperature and maximum number of tokens, system and custom hints for each part, test cases used and the final ranked ELO rating for each candidate hint.

7. Portkey : An additional tool for logging and tracking all hint chains and their responses.
   
   

Installation and use

To use gpt-prompt-engineer, users must configure the tool in Jupyter or Google Colab. They should add their OpenAI API key to the line openai.api_key = ""ADD YOUR KEY HERE"" and, if using the Claude 3 version, add their Anthropic API key to the line ANTHROPIC_API_KEY = ""ADD YOUR KEY HERE"" .

How to use

The tool is designed with ease of use in mind, with a simple process for creating and testing tooltips:

1. Define your use case and test cases. For example, if you want to create a title for a landing page, your description could be “Given a prompt, generate a title for a landing page,” and your test cases could include specific prompts such as “Promoting the innovative new fitness app Smartly.” or “Why a vegetarian diet is good for your health.”

2. Select the number of tips to generate. Keep in mind that this can get expensive if you generate a lot of tips, so 10 is a good starting point.

3. Call generate_optimal_prompt(description, test_cases, number_of_prompts) to generate a list of potential prompts and test and evaluate their performance.

4. The final ELO ratings will be printed in a table, sorted in descending order. The higher the rating, the better the hint.
   
   

Contribution and future development

gpt-prompt-engineer project is open to contributions and welcomes ideas to expand its capabilities. Some potential areas for development include:

• Availability of several different system tooltip generators, creating different styles of tooltips to cover more tasks (e.g. examples, long-form, short, markdown, etc.).

• Automatic generation of test cases.

• Extending the classification version to support more than two classes using tiktoken