stoRy time with Shiny, Quarto, and Google Cloud Run

Write and Illustrate Stories with AI

Umair Durrani

https://dru.quarto.pub/slides-3f3c/

Everyone enjoys stories

AI is great at making up stuff

Source 1; Source 2

AI is great at making up stuff creating stories 🙂

What is stoRy time?

A web application that:

  • takes the first sentence of a story you want to write
  • uses generative AI models to create a full story with images

Why R in stoRy time?

Let’ take a look at stoRy time

App Link

You can also change the slide theme

You can also change the slide theme

Who am I

  • Umair Durrani

  • PhD and Postdoc from University of Windsor

  • Data Scientist at Presage Group

    • Develop apps
    • Analyze and present data

Who are you

Developer


User

How does stoRy time work? and why you should care?

  • You learn a lot of useful stuff working on such project:

    • Making REST API requests
    • Develop web applications
    • Create parameterized documents
    • Working with Cloud

Step 1: Create story text

How to generate a story with AI?

Several text generation models exist.

I used llama-3.1-8b-instruct from Cloudflare Workers AI Models API.

What’s an API?

An API is a programmatic way to interact with a webservice that allows us to automate the retrieval of data.

  • GET
  • POST

I used the httr2 R package to make HTTP requests to Cloudflare Workers AI Models endpoint

Source: Talk by Ted Laderas

Function to generate story

get_story <- function(prompt,
                      num_of_sentences = 5,
                      max_tokens = 1000,
                      ACCOUNT_ID = Sys.getenv("ACCOUNT_ID"),
                      API_KEY = Sys.getenv("API_KEY"),
                      base_url = cf_base_url()){

  if (is.null(prompt) | num_of_sentences < 3){
    return(NULL)
  }

  if (test_profanity(prompt)){
    return(NULL)
  }

  url_txt <- paste0(base_url, ACCOUNT_ID, "/ai/run/@cf/meta/llama-3.1-8b-instruct")

  # Make an API request
  response_text <- httr2::request(url_txt) |>
    httr2::req_headers(
      "Authorization" = paste("Bearer", API_KEY)
    ) |>
    httr2::req_body_json(list(
      max_tokens = max_tokens,
      messages = list(
        list(role = "system",
             content = paste0("You tell short stories.
             Each sentence must describe all details.
             Each story must have ",  num_of_sentences,  " sentences.
             The story must have a beginning, a climax and an end.")),
        list(
          role = "user",
          content = prompt
        )
      ))) |>
    httr2::req_method("POST") |>
    httr2::req_error(is_error = \(resp) FALSE) |>
    httr2::req_perform() |>
    httr2::resp_body_json()

  # If response is successful, append it to the user prompt
  # clean it, and split the text into 5 sentences
  if (isTRUE(response_text$success)){
    full_text <- response_text$result$response #paste(prompt, response_text$result$response)
    cleaned_text <- gsub("\n", "", full_text)
    split_text <- unlist(strsplit(cleaned_text, "(?<=[.])\\s*(?=[A-Z])", perl = TRUE))
  } else {
    split_text <- NULL
  }

  # c(prompt, split_text)
  split_text
}

Test the function

new_story <- get_story(
  prompt = "There once was a prince in the land of Persia.",
  num_of_sentences = 3
)

Generates:

[1] "He wore a intricately designed golden crown, adorned with precious rubies and diamonds that caught the light of a full moon, and from the moment he was born, he was destined for greatness."                                               
[2] "On his seventh name-day, the prince rode his white stallion, Majdool, through the crowded market of Isfahan, where merchants in tunics and turbans waved in reverence as he passed by, their faces pressed against the walls of their stalls, watching the prince's stately procession."
[3] "As he approached the grand square, the prince's horse let out a high-pitched whinny, and the prince's mother, the queen, gently corrected the animal with a soft voice, though her eyes were fixed adoringly on her son."

Step 2: Create images for illustration

Use Stable Diffusion to generate images

req_single_image <- function(prompt,
                             instructions,
                             ACCOUNT_ID = Sys.getenv("ACCOUNT_ID"),
                             API_KEY = Sys.getenv("API_KEY"),
                             base_url = cf_base_url()){

  url_img <- paste0("https://api.cloudflare.com/client/v4/accounts/", ACCOUNT_ID, "/ai/run/@cf/bytedance/stable-diffusion-xl-lightning")

  # Create the request
  httr2::request(url_img) |>
    httr2::req_headers(
      "Authorization" = paste("Bearer", API_KEY)
    ) |>
    httr2::req_body_json(list(prompt = paste0(
      prompt, " ",
      instructions
    ))) |>
    httr2::req_method("POST")
}

# Get image if request is successful
get_image <- function(response){
  if (response$status_code == 200){
    png_img <- httr2::resp_body_raw(response)
  } else{
    png_img <- NULL
  }
  png_img
}

Generate some images

image_prompt <- "This scene should be illustrated ..."
reqs <- lapply(
  new_story,
  function(x){
    req_single_image(x, image_prompt)
  }
)
resps <- httr2::req_perform_parallel(reqs, on_error = "continue")
# All images
new_all_imgs <- lapply(resps, get_image)

Step 3: Creating Slides

Quarto is a new, open-source,
scientific and technical
publishing system

A schematic representing the multi-language input (e.g. Python, R, Observable, Julia) and multi-format output (e.g. PDF, html, Word documents, and more) versatility of Quarto.

Source

Quarto

An open-source scientific and technical publishing system

Quarto Gallery

I used parameters in the quarto file and the quarto R package.

Quarto file (.qmd)

YAML options (instructions):

---
title: "stoRy time with shiny and quarto"
author: "A story written by you & AI"
format: 
    revealjs:
      embed-resources: true
      center: true
      transition: slide
      background-transition: fade
params:
  story_prompt: ""
  story: ""
  imgs: ""
---
    
"<Use story and images here for creating slides>"

quarto R package

quarto::quarto_render(
  input = "<QUARTO FILE>",
  output_format = "all",
  metadata = list(
    theme = "<REVEAL JS THEME>",
    "title-slide-attributes" = list(
      "data-background-image" = paste0("data:image/png;base64,", base64enc::base64encode(utils::tail(new_all_imgs, 1)[[1]])),
      "data-background-size" = "cover",
      "data-background-opacity" = 0.3
    )
  ),
  quarto_args = c(
    "--metadata",
    paste0("title=", "<STORY TITLE>")
  ),
  execute_params = list(
    story_prompt = "<STORY PROMPT>",
    story = "<STORY TEXT>",
    imgs = lapply(new_all_imgs, base64enc::base64encode)
  )
)

Reveal Themes

Step 4: Create a web app

Shiny web application

Make web apps with shiny

Shiny playground

shinylive

Shiny web application

App Link

Source: The Anatomy of a Shiny Application

Step 5: Deploy the application

Google Cloud Run

Docker Containers

“It runs on my computer”

A container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another.

Source

Dockerfile

For an example shiny app:

# Base R Shiny image
FROM rocker/shiny

# Make a directory in the container
RUN mkdir /home/shiny-app

# Install R dependencies
RUN R -e "install.packages(c('dplyr', 'ggplot2', 'gapminder'))"

# Copy the Shiny app code
COPY app.R /home/shiny-app/app.R

# Expose the application port
EXPOSE 8180

# Run the R Shiny app
CMD Rscript /home/shiny-app/app.R

How To Run Shiny Apps in a Docker Container; Image source

Push to GitHub

GitHub Repo for stoRytime

App Link

Google Cloud Run deployment

Step 6: Celebrate 🎉

Questions and Contact

  • umairdurrani.com
  • @transport-talk.bsky.social

Try the app