Read and Write Files in Java Using Separate Threads

1. Introduction

When it comes to file handling in Java, it can be challenging to manage large files without causing performance issues. That’s where the concept of using separate threads comes in. By using separate threads, we can efficiently read and write files without blocking the main thread. In this tutorial, we’ll explore how to read and write files using separate threads.

2. Why Use Separate Threads

Using separate threads for file operations can improve performance by allowing concurrent execution of tasks. In a single-threaded program, file operations are performed sequentially. For example, we read the entire file first and then write to another file. This can be time-consuming, especially for large files.

By using separate threads, multiple file operations can be performed simultaneously, taking advantage of multicore processors and overlapping I/O operations with computation. This concurrency can lead to better utilization of system resources and reduced overall execution time. However, it’s essential to note that the effectiveness of using separate threads depends on the nature of the tasks and the I/O operations involved.

3. Implementation of File Operations Using Threads

Reading and writing files can be done using separate threads to improve performance. In this section, we’ll discuss how to implement file operations using threads.

3.1. Reading Files in Separate Threads

To read a file in a separate thread, we can create a new thread and pass a Runnable object that reads the file. The FileReader class is used to read a file. Moreover, to enhance the file reading process, we use a BufferedReader that allows us to read the file line by line efficiently:

Thread thread = new Thread(new Runnable() {
    @Override
    public void run() {
        try (BufferedReader bufferedReader = new BufferedReader(new FileReader(filePath))) {
            String line;
            while ((line = bufferedReader.readLine()) != null) {
                System.out.println(line);
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
});

thread.start();

3.2. Writing Files in Separate Threads

We create another new thread and use the FileWriter class to write data to the file:

Thread thread = new Thread(new Runnable() {
    @Override
    public void run() {
        try (FileWriter fileWriter = new FileWriter(filePath)) {
            fileWriter.write("Hello, world!");
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
});

thread.start();

This approach allows reading and writing to run concurrently, meaning they can happen simultaneously in separate threads. This is particularly beneficial when one operation doesn’t depend on the completion of the other.

4. Handling Concurrency

Concurrent access to files by multiple threads requires careful attention to avoid data corruption and unexpected behavior. In the earlier code, the two threads are started concurrently. This means that both can execute simultaneously, and there is no guarantee about the order in which their operations will be interleaved. If a reader thread tries to access the file while a write operation is still ongoing, it might end up reading incomplete or partially written data. This can result in misleading information or errors during processing, potentially affecting downstream operations that rely on accurate data.

Moreover, if two writing threads simultaneously attempt to write data to the file, their writes might interleave and overwrite portions of each other’s data. Without proper synchronization handling, this could result in corrupted or inconsistent information.

To address this, one common approach is to use a producer-consumer model. One or more producer threads read files and add them to a queue, and one or more consumer threads process the files from the queue. This approach allows us to easily scale our application by adding more producers or consumers as needed.

5. Concurrent File Processing With BlockingQueue

The producer-consumer model with a queue coordinates operations, ensuring a consistent order of reads and writes. To implement this model, we can use a thread-safe queue data structure, such as a BlockingQueue. The producers can add files to the queue using the offer() method, and the consumers can retrieve files using the poll() method.

Each BlockingQueue instance has an internal lock that manages access to its internal data structures (linked list, array, etc.). When a thread attempts to perform an operation like offer() or poll(), it first acquires this lock. This ensures that only one thread can access the queue at a time, preventing simultaneous modifications and data corruption.

By using BlockingQueue, we decouple the producer and consumer, allowing them to work at their own pace without directly waiting for each other. This can improve overall performance.

5.1. Create FileProducer

We begin by creating the FileProducer class, representing the producer thread responsible for reading lines from an input file and adding them to a shared queue. This class utilizes a BlockingQueue to coordinate between the producer and consumer threads. It accepts a BlockingQueue to serve as a synchronized storage for lines, ensuring that the consumer thread can access them.

Here is an example of the FileProducer class:

class FileProducer implements Runnable {
    private final BlockingQueue<String> queue;
    private final String inputFileName;

    public FileProducer(BlockingQueue<String> queue, String inputFileName) {
        this.queue = queue;
        this.inputFileName = inputFileName;
    }
    // ...
}

Next, in the run() method, we open the file using BufferedReader for efficient line reading. We also include error handling for potential IOException that might occur during file operations.

@Override
public void run() {
    try (BufferedReader reader = new BufferedReader(new FileReader(inputFileName))) {
        String line;
        // ...
    } catch (IOException e) {
        e.printStackTrace();
    }
}

After we open the file, the code enters a loop, reading lines from the file and concurrently adding them to the queue using the offer() method:

while ((line = reader.readLine()) != null) {
    queue.offer(line);
}

5.2. Create FileConsumer

Following that, we introduce the FileConsumer class, which represents the consumer thread tasked with retrieving lines from the queue and writing them into an output file. This class accepts a BlockingQueue as input for receiving lines from the producer thread:

class FileConsumer implements Runnable {
    private final BlockingQueue<String> queue;
    private final String outputFileName;

    public FileConsumer(BlockingQueue queue, String outputFileName) {
        this.queue = queue;
        this.outputFileName = outputFileName;
    }
    
    // ...
}

Next, in the run() method we use BufferedWriter to facilitate efficient writing to the output file:

@Override
public void run() {
    try (BufferedWriter writer = new BufferedWriter(new FileWriter(outputFileName))) {
        String line;
        // ...
    } catch (IOException e) {
        e.printStackTrace();
    }
}

After we open the output file, the code enters a continuous loop, using the poll() method to retrieve lines from the queue. If a line is available, it writes the line to a file. The loop terminates when poll() returns null, indicating that the producer has finished writing lines and there are no more lines to process:

while ((line = queue.poll()) != null) {
    writer.write(line);
    writer.newLine();
}

5.3. Orchestrator of Threads

Finally, we wrap everything together within the main program. First, we create a LinkedBlockingQueue instance to serve as the intermediary for lines between the producer and consumer threads. This queue establishes a synchronized channel for communication and coordination.

BlockingQueue<String> queue = new LinkedBlockingQueue<>();

Next, we create two threads: a FileProducer thread responsible for reading lines from the input file and adding them to the queue. We also create a FileConsumer thread tasked with retrieving lines from the queue and expertly handling their processing and output to the designated output file:

String fileName = "input.txt";
String outputFileName = "output.txt"

Thread producerThread = new Thread(new FileProducer(queue, fileName));
Thread consumerThread = new Thread(new FileConsumer(queue, outputFileName);

Subsequently, we initiate their execution using the start() method. We utilize the join() method to ensure both threads gracefully finish their work before the program bows out:

producerThread.start();
consumerThread.start();

try {
    producerThread.join();
    consumerThread1.join();
} catch (InterruptedException e) {
    e.printStackTrace();
}

Now, let’s create an input file and then run the program:

Hello,
Baeldung!
Nice to meet you!

After running the program, we can inspect the output file. We should see the output file contains the same lines as the input file:

Hello,
Baeldung!
Nice to meet you!

In the provided example, the producer is adding lines to the queue in a loop, and the consumer is retrieving lines from the queue in a loop. This means multiple lines can be in the queue simultaneously, and the consumer may process lines from the queue even as the producer is still adding more lines.

6. Conclusion

In this article, we’ve explored the utilization of separate threads for efficient file handling in Java. We also demonstrated using BlockingQueue to achieve synchronized and efficient line-by-line processing of files.