AI - Local or Cloud ?

If you are integrating AI into your applications, there are three ways to do it.

API integration

You integrate with ChatGPT, Cohere, or a similar service. You get an API key and start using APIs offered by these services. 

Pros 

1. Access to perhaps the best models known to humanity

2. High accuracy and speed

3. No maintenance

Cons

1. Cost

2. Privacy - Not all of your users will be comfortable with their data being shared with third party services.

Bring your own cloud

You host models like Llama2, Mistral, etc. on your own GPU servers.

Pros

1. You can train the models on your own data. Unless you are a huge company sitting on volumes of data, this is not really an advantage.

Cons

1. Privacy - This is lesser of a concern than sending your private data to third party sites over the internet. But again, all of your users might not be comfortable with you being able to read their data on your servers.

2. Cost - GPUs are very costly. Running your own models in the cloud is quite a costly affair at this time.

3. Maintenance - You are responsible for maintaining your servers and models.

Browser

You run the models in your browser.

Pros

1. Privacy - User data doesn’t have to leave the user’s machine.

2. Cost - Since all the computation happens on the user’s machine, there is no infrastructure cost.

3. Maintenance - There is no server maintenance.

Cons

1. Not so powerful models

2. Less accurate and slow

3. Large memory footprint and CPU intensive - A lot of effort is required to manage a smooth user experience.

It is an exciting time. With browser technology advancing rapidly and WASM bringing the power of the byte code to browsers, things will become a lot more exciting for browser based AI.