In this tutorial I will show you how to easily scan the contents of files using Regular Expressions (RegEx) using the Rust language. Rust implements a regular expression engine similar to many other regex engines such as Perl Compatible Regular Expressions (PCRE) and ECMAScript but lacks features such as look-arounds and backreferences. You can visit the source code of the regex crate by visiting the Rust implementation of regular expressions on GitHub.

Problem

Picture a situation where you find yourself in the position of needing to extract URLs from a file. Our motivation could be twofold – either to scrutinize these URLs for any signs of malicious content or threats, or perhaps, to compile a set of metadata pertaining to network connections from log files. To accomplish this undertaking with precision and efficiency, we can turn to the powerful capabilities of Rust, leveraging its regular expression functionality.

Installing the regex Crate

We can install the regex crate by simply using using either of the following methods. Using the first method we can use the cargo add command.

This will download and the latest version of the regex crate which we can use in our Rust application. If you would like to specify a different version you can do so by adding the specific version to your Cargo.toml file.

Once you’ve used either of the following method your Cargo.toml file should look similar to the following.

Importing Error, File, BufRead, & Read

In order to handle the processing of our file we need to import some Rust structures and traits that allow us to handle files and data gracefully.

Let’s break up the following imports to understand it’s functionality.

This line imports the Error trait which is essential for error handling in Rust.

This line imports the File struct from the std::fs module. The File struct enables file operations such as opening, creation, reading, and writing files.

This line imports the the io module itself, which is essential for input and output (IO) operations in Rust. Additionally, it imports the BufRead trait from io, which allows the buffered reading of data. Finally, the Read trait, is used for reading bytes.

Importing the Regex Crate

Before we can begin using the regex crate we need to import the regex crate into our Rust projects main.rs file.

This line imports the Regex struct and its methods within our Rust program.

Writing the Search Method

Once we have imported the Rust traits and structures we will use to search files with regex we can proceed to write the function to search files. We’ll create the function search_file_for_pattern, which will take two input parameters: in_file_str, which is our file we want to scan as a string. The re_str represents our regular expression pattern. Finally this function will return a Result type object or an Error.

Next we will compile the re_str into a new regular expression object and store it into a variable.

Next we will open our input file using and store the handle in a variable.

Next we’ll create a new buffer object as a vector and read the contents of the input file.

Next we need to loop over the contents of our buffer object, searching for new line characters. If encountered we’ll assume the end of a line and keep track of that line. After which we’ll convert the slice of bytes between the start and index before into a UTF-8 string with std::str::from_utf8.

After we have our line as a string we’ll scan that line with our regular expression object and if a match is found we will print that matched value to the console.

After processing the line we’ll update the start position for the next line.

In order to handle the very last line of the file we need to also check to see if there are any remaining bytes in our buffer. We can easily check this condition and handle the last line with a check on the start variable and buffer.len.

Below is our completed function which contains all the logic in order to read our input file, compile our regex, read the the input file to a buffer, and scan each line with our regex value.

Using Our Search Function

Once we have completed our search_file_for_pattern function we can call the function inside our main rust function. In this example I’ve defined a variable called file_to_scan which is the string representation of the file I want to scan, I’m using a local HTML file. Next, I’ve defined the re variable which is our regex pattern. This regex pattern is looking for URLs inside of strings. Using this logic we will be able to extract all URL’s from our input file using rust!

Using this logic we will be able to extract all URL’s from our input file using rust!

GitHub Source

You can find the full source code for this project on GitHub.

Conclusion

In conclusion, we have explored how to harness the power of regular expressions (RegEx) in the Rust programming language to scan the contents of files. While Rust's regex engine may lack some advanced features found in other engines, it remains a robust tool for many common use cases.

Throughout this tutorial, we learned how to use RegEx patterns to extract specific information from files, such as URLs, which can be invaluable for tasks like analyzing URLs for security threats or gathering metadata from network log files. By mastering these techniques, you have equipped yourself with a valuable skill that can be applied to a wide range of data processing and analysis tasks.

Remember that Rust's ecosystem offers various libraries and tools, including the regex crate, which further enhances the capabilities of regex in Rust. Whether you are a seasoned Rust developer or just starting your journey with the language, regex in Rust can be a powerful tool in your toolkit for working with text data effectively.

As you continue to explore Rust and its versatile features, you will find numerous opportunities to leverage regular expressions to solve real-world problems efficiently. So, keep practicing, experimenting, and building, and you'll be well-prepared to tackle data manipulation challenges using Rust and RegEx.

If you found this how-to guide helpful and would like to stay updated on more Rust-related tips, tricks, and tutorials, please consider following me on social media. Whether you have further questions or need assistance with any aspect of this process, feel free to reach out – I'm here to help you on your Rust journey!