How to create static variable that can reference different static strings?

Question

I'm just getting up to speed with rust. As an exercise I'm trying to recreate strtok() in rust. However I'm stuck with correcting the types..

Here's what I have (Live Demo):

fn strtok(str: Option<&'static str>, delimiter: char) -> &'static str {
    // Remember the string slice for next time
    let temp_str = "";
    let mut static_view: &mut &'static str = temp_str;
    if str.is_some() {
        static_view = str.unwrap();
    }
    // Get token from start to delimiter 
    let index = static_view.find(delimiter).unwrap();
    let token = &static_view[..index];
    static_view = &static_view[index..];
    token
}

fn main() {
    let str: &str = "Hello, World";
    let token1 = strtok(Some(str), ',');
    let token2 = strtok(None, ',');

    println!("{}", token1);
    println!("{}", token2);
}

I wanted to create a mutable static reference to a static string (slice), so I could switch the string slice for a different one if provided. If not, the function should go off the last known string slice.

However, I get this error:

    error[E0308]: mismatched types
 --> src/main.rs:5:46
  |
5 |     let mut static_view: &mut &'static str = temp_str;
  |                          -----------------   ^^^^^^^^ types differ in mutability
  |                          |
  |                          expected due to this
  |
  = note: expected mutable reference `&mut &'static _`
                     found reference `&_`

Which I don't really understand. I never asked for the slice to be mutable, but I want the reference to it be mutable so I can reassign it. How should

Answer 1

There are several ways to have shared mutable state, with different tradeoffs. Rust makes you choose.

• You can have a static mut global variable, like this:

  static mut STRTOK_BUF: &str = "";
  

  However, using such a variable requires unsafe code, because (among other problems) Rust can't rule out data races when multiple threads access it.

• You can have a safe static of type Mutex<&str>.

• You can use a thread-local variable of type Cell<&str>.

However, this particular example works out a little nicer in Rust if you avoid shared mutable state.

---

There are several ways of framing strtok's functionality as a Rust function.

• If you wanted to implement the C function strtok exactly in Rust, it would be an unsafe fn with raw pointer arguments, and most likely it would use a static mut global variable of type *mut c_char.

• If you wanted to write perfectly idiomatic Rust, you'd make this return an iterator, like str::split.

• If you want something in between, try implementing strtok_r instead. Then at least there is not a global mutable variable. Rather, the caller provides the variable. The code below offers a safe way to implement that (playground). This also lets you use the function with strings of any lifetime.

pub fn strtok_r<'s>(s: Option<&'s str>, delim: char, ptrptr: &mut &'s str) -> Option<&'s str> {
    let s = s.unwrap_or(*ptrptr).trim_start_matches(delim);
    match s.find(delim) {
        Some(i) => {
            *ptrptr = &s[i..];
            Some(&s[..i])
        }
        None => {
            *ptrptr = "";
            if s.is_empty() {
                None
            } else {
                Some(s)
            }
        }
    }
}

Answer 2

As @kmdreko said, strtok() is a very bad function to recreate in Rust (and a very bad function in general) since it relies on global mutable state, and can easily cause UB.

If you really want, here is how it would look like in unsafe code (note this is not equivalent to the C API, since it requires &'static str):

/// # Safety
///
/// You really must not call this on multiple threads concurrently.
unsafe fn strtok(s: Option<&'static str>, delimiter: char) -> &'static str {
    // Remember the string slice for next time
    static mut PREV_STRING: &'static str = "";
    if let Some(s) = s {
        PREV_STRING = s;
    }
    let s = PREV_STRING;
    // Get token from start to delimiter 
    let (token, new_string) = match s.find(delimiter) {
        Some(index) => (&s[..index], &s[index + delimiter.len_utf8()..]),
        None => (s, ""),
    };
    PREV_STRING = new_string;
    token
}

And here is how it looks like with a safe API:

use std::sync::atomic::{AtomicUsize, Ordering};
use std::sync::RwLock;

fn find_string_parts(s: &str, start: usize, delimiter: char) -> (&str, usize) {
    match s[start..].find(delimiter) {
        Some(index) => (&s[start..index], start + index + delimiter.len_utf8()),
        None => (&s[start..], s.len()),
    }
}

fn strtok(s: Option<&'static str>, delimiter: char) -> &'static str {
    // Remember the string slice for next time
    static PREV_STRING: RwLock<&str> = RwLock::new("");
    static POSITION_IN_STRING: AtomicUsize = AtomicUsize::new(0);
    if let Some(s) = s {
        let (token, new_position) = find_string_parts(s, 0, delimiter);
        let mut prev_string = PREV_STRING.write().unwrap();
        *prev_string = s;
        POSITION_IN_STRING.store(new_position, Ordering::SeqCst);
        token
    } else {
        let prev_string = PREV_STRING.read().unwrap();
        let s = *prev_string;
        let mut token = "";
        _ = POSITION_IN_STRING.fetch_update(
            Ordering::SeqCst,
            Ordering::SeqCst,
            |position_in_string| {
                let new_position;
                (token, new_position) = find_string_parts(s, position_in_string, delimiter);
                Some(new_position)
            },
        );
        token
    }
}