My main issue with using the general chatbot is that it’s an incredibly inefficient way to convey information. For writing tasks I essentially need to type most of the answer first to get reasonable outputs when considering my actual constraints.
More specialized tooling will have these constraints built-in, which will increase productivity.
Even if we have the perfect general chatbot, it’s still a lot of work to concisely describe your requirements to it.
I agree. My 3070 runs the 8B Llama3 model in about 250ms, especially for short responses.