AI for Web Devs: What Are Neural Networks, LLMs, & GPTs?

Welcome back to

/** @type {import('@builder.io/qwik-city').RequestHandler} */

export const onPost = async (requestEvent) => {

  const OPENAI_API_KEY = requestEvent.env.get('OPENAI_API_KEY')

  const formData = await requestEvent.parseBody()



  const prompt = formData.prompt

  const body = {

    model: 'gpt-3.5-turbo',

    messages: [{ role: 'user', content: prompt }]

  }



  const response = await fetch('https://api.openai.com/v1/chat/completions', {

    // ... fetch options

  })

  const data = await response.json()



  const responseBody = data.choices[0].message.content



  requestEvent.send(new Response(responseBody))

}

So today I’ll just share the snippet, which I put inside a utils.js file in the root of my project. This jsFormSubmit function accepts an request based on the form attributes and returns the resulting handler. Sweet!

As for the reactive data, Qwik provides two options, . I prefer useStore, which allows us to create an object whose properties are attribute. By default, an HTML form will submit data to the current URL, so we only need to set the , we need to wrap an event handler inside Qwik’s $ function.

Inside the event handler, we’ll want to clear out any previous data from state.text, set state.isLoading to true, then pass the form’s DOM node to our fancy jsFormSubmit function. This should submit the HTTP request for us. Once it comes back, we can update state.text with the response body, and return state.isLoading to false.

const handleSubmit = $(async (event) => {

  state.text = ''

  state.isLoading = true



  /** @type {HTMLFormElement} */

  const form = event.target



  const response = await jsFormSubmit(form)



  state.text = await response.text()

  state.isLoading = false

})

OK! We should now have a client-side form that uses JavaScript to submit an HTTP request to the server while tracking the loading and response states, and updating the UI accordingly.

That was a lot of work to get the same solution we had before but with fewer features. BUT the key benefit is we now have direct access to the platform primitives we need to support streaming.

Enable Streaming on the Server

Before we start streaming responses from OpenAI, I think it’s helpful to start with a very basic example to get a better grasp of streams. Streams allow us to send small chunks of data over time. So as an example, let’s print out some iconic David Bowie lyrics in tempo with the song, “ using the . This optional parameter can be an object with a start method that’s called when the stream is constructed.

The start method is responsible for the steam’s logic and has access to the stream controller, which is used to send data and close the stream.

const stream = new ReadableStream({

  start(controller) {

    // Stream logic goes here

  }

})

OK, let’s plan out that logic. We’ll have an array of song lyrics and a function to ‘sing’ them (pass them to the stream). The sing function will take the first item in the array and pass that to the stream using the controller.enqueue() method. If it’s the last lyric in the list, we can close the stream with controller.close(). Otherwise, the sing method can call itself again after a short pause.

const stream = new ReadableStream({

  start(controller) {

    const lyrics = ['Ground', ' control', ' to major', ' Tom.']

    function sing() {

      const lyric = lyrics.shift()



      controller.enqueue(lyric)



      if (lyrics.length < 1) {

        controller.close()

      } else {

        setTimeout(sing, 1000)

      }

    }

    sing()

  }

})

So each second, for four seconds, this stream will send out the lyrics “Ground control to major Tom.” Slick!

Because this stream will be used in the body of the Response, the connection will remain open for four seconds until the response completes. But the frontend will have access to each chunk of data as it arrives, rather than waiting the full four seconds.

This doesn’t speed up the total response time (in some cases, streams can increase response times), but it does allow for a faster perceived response, and that makes a better user experience.

Here’s what my code looks like:

/** @type {import('@builder.io/qwik-city').RequestHandler} */

export const onPost: RequestHandler = async (requestEvent) => {

  const stream = new ReadableStream({

    start(controller) {

      const lyrics = ['Ground', ' control', ' to major', ' Tom.']

      function sing() {

        const lyric = lyrics.shift()



        controller.enqueue(lyric)



        if (lyrics.length < 1) {

          controller.close()

        } else {

          setTimeout(sing, 1000)

        }

      }

      sing()

    }

  })



  requestEvent.send(new Response(stream))

}

Unfortunately, as it stands right now, the client will still be waiting four seconds before seeing the entire response, and that’s because we weren’t expecting a streamed response.

Let’s fix that.

Enable Streaming on the Client

Even when dealing with streams, the default browser behavior when receiving a response is to wait for it to complete. In order to get the behavior we want, we’ll need to use client-side JavaScript to make the request and process the streaming body of the response.

We’ve already tackled that first part inside our handleSubmit function. Let’s start processing that response body.

We can access the ReadableStream from the response body’s method that we can use to access the next chunk of data, as well as the information if the response is done streaming or not.

The only ‘gotcha’ is that the data in each chunk doesn’t come in as text, it comes in as a .

Ok, that’s a lot of theory. Let’s break down the logic and then look at some code.

When we get the response back, we need to:

1. Grab the reader from the response body using response.body.getReader()




2. Setup a decoder using TextDecoder and a variable to track the streaming status.



3. 
   

   Process each chunk until the stream is complete, with a while loop that does this:

   
   
   
   
   
   1. Grab the next chunk’s data and stream status.
   
   
   2. 
      Decode the data and use it to update our app’s state.text.
   
   
   3. Update the streaming status variable, terminating the loop when complete.
   
   
   
   
   
   



4. Update the loading state of the app by setting state.isLoading to false.

The new handleSubmit function should look something like this:

const handleSubmit = $(async (event) => {

  state.text = ''

  state.isLoading = true



  /** @type {HTMLFormElement} */

  const form = event.target



  const response = await jsFormSubmit(form)



  // Parse streaming body

  const reader = response.body.getReader()

  const decoder = new TextDecoder()

  let isStillStreaming = true



  while(isStillStreaming) {

    const {value, done} = await reader.read()

    const chunkValue = decoder.decode(value)



    state.text += chunkValue



    isStillStreaming = !done

  }



  state.isLoading = false

})

Now, when I submit the form, I see something like:

“Ground

control

to major

Tom.”

Hell yeah!!!

OK, most of the work is down. Now we just need to replace our demo stream with the OpenAI response.

Stream OpenAI Response

Looking back at our original implementation, the first thing we need to do is modify the request to OpenAI to let them know that we would like a streaming response. We can do that by setting the module on NPM did not properly support streaming responses. That issue has been fixed, and I think a better solution would be to use that module and pipe their data through a to grab the rest of the string after “data: “.