You can run using your own OpenAI key by setting 2 environment variables:
(1) SKETCH_USE_REMOTE_LAMBDAPROMPT=False
(2) OPENAI_API_KEY=YOUR_API_KEY
To run entirely locally (using your own GPU and a model like Bloom) one would have to add a new prompt type to `lambdaprompt` (the package that this depends on), have a machine with enough GPU resources, and then add a slight modification to sketch.
Not sure if this is a business you're building out of this or an experiment. For real use for any of my customers, I would need to run this entirely locally.
I think it's really awesome though!
Curious what "enough GPU resources" looks like? Would a GeForce RTX 40 or 30 series card with 12-24GB of RAM be sufficient per user running locally on their machine?
If this is using OpenAI which it seems is what it is using, It is only sending column headers / column names. Not the data. If you are concerned about column names, you could also mask it on the way out and back in. If you are looking for an end to end database connect and query, please reach out to me.
