Copy a File or Directory in Java

Copy a File or Directory in Java

Copy a file or directory in Java is a common operation. In this post, we will go through different options to perform following operation.

  1. Copy a file from one location to another location.
  2. Copy all content of directory from one location to another.

We will be using Java 7+ and Apache Commons API for copying a file or directory in Java.

1.  Copy Directory Using Java

We will be using Files.copy(source, destination, StandardCopyOption.REPLACE_EXISTING) to copy files /directory from the source location to destination location.

 


public class CopyDirectory {

    public static void main(String[] args) throws IOException {

        Path sourceDirectory = Paths.get("/Users/personal/tutorials/source");
        Path targetDirectory = Paths.get("/Users/personal/tutorials/target");

        //copy source to target using Files Class
        Files.copy(sourceDirectory, targetDirectory);

    }

}

This will copy source directory to target location. We need to remember following points

  1. If target directory exists, the system will throw FileAlreadyExistsException.
  2. Use StandardCopyOption.REPLACE_EXISTING to replace target location with the source.
  3. Above method will only copy source directory but not files or subdirectories.
  4. To copy files and subdirectories, use Files.walkFileTree method.

 

2. Copy File Using Java

We will be using Files.copy(source, destination, StandardCopyOption.REPLACE_EXISTING) to copy files from source location to destination location.

 


public class CopyFIle {

    public static void main(String[] args) throws IOException {

        Path sourceDirectory = Paths.get("/Users/umesh/personal/tutorials/source/Variation_Relations.csv");
        Path targetDirectory = Paths.get("/Users/umesh/personal/tutorials/target/Variation_Relations.csv");

        //copy source to target using Files Class
        Files.copy(sourceDirectory, targetDirectory);

    }
}

Above assume that target folder already exists, if “target” directory does not exist, the system will throw NoSuchFileException.

This is not what we might want in many cases. Most of the time we want to Copy files from one directory to another in Java including sub directories.

We will be usingFiles.walkFileTree from NIO Java7 approach to recursively copy directory and all files to another location.

Custom Directory Walker


import java.io.IOException;
import java.nio.file.FileAlreadyExistsException;
import java.nio.file.FileSystemLoopException;
import java.nio.file.FileVisitResult;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.SimpleFileVisitor;
import java.nio.file.attribute.BasicFileAttributes;

import static java.nio.file.FileVisitResult.CONTINUE;
import static java.nio.file.FileVisitResult.SKIP_SUBTREE;

/
public class CustomFileVisitor extends SimpleFileVisitor<Path> {

    final Path source;
    final Path target;

    public CustomFileVisitor(Path source, Path target) {
        this.source = source;
        this.target = target;
    }



    @Override
    public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs)
            throws IOException
    {
       
        Path newDirectory= target.resolve(source.relativize(dir));
        try{
            Files.copy(dir,newDirectory);
        }
        catch (FileAlreadyExistsException ioException){
            //log it and move
            return SKIP_SUBTREE; // skip processing
        }

        return FileVisitResult.CONTINUE;
    }

    @Override
    public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {

       Path newFile = target.resolve(source.relativize(file));

        try{
            Files.copy(file,newFile);
        }
        catch (IOException ioException){
            //log it and move
        }

        return FileVisitResult.CONTINUE;

    }

    @Override
    public FileVisitResult postVisitDirectory(Path dir, IOException exc) throws IOException {

        
        return FileVisitResult.CONTINUE;
    }

    @Override
    public FileVisitResult visitFileFailed(Path file, IOException exc) {
        if (exc instanceof FileSystemLoopException) {
            //log error
        } else {
            //log error
        }
        return CONTINUE;
    }
}

Time to Use our CustomFileVisitor


public class CopyFilesRecursively {

    public static void main(String[] args) throws IOException {

        Path sourceLocation= Paths.get("/Users/umesh/personal/tutorials/source");
        Path targetLocation =Paths.get("/Users/umesh/personal/tutorials/target");

        CustomFileVisitor fileVisitor = new CustomFileVisitor(sourceLocation, targetLocation);
        //You can specify your own FileVisitOption
        Files.walkFileTree(sourceLocation, fileVisitor);
    }
}

If you want to permissions file permissions while copy directories/file using Files.copy(), look out for StandardCopyOption.html#COPY_ATTRIBUTES

2. Copy File Using Apache Commons

Apache Commons provides more easy and clean way to copy files/directories


public class CopyDirectoryApache {

    public static void main(String[] args) throws IOException {

        File sourceLocation= new File("/Users/umesh/personal/tutorials/source");
        File targetLocation = new File("/Users/umesh/personal/tutorials/target");

        FileUtils.copyDirectory(sourceLocation, targetLocation);
    }

In this post, we learned following ways Copy a File or Directory in Java

  1. Use Files.copy for simple operation.
  2. Use custom FileVisitor to recursively copy file or directories.
  3. Use Apache Commons IO.

 

Read our other post to learn How to create a directory in Java.

All the code of this article is available Over on Github. This is a Maven-based project.

References

  1. Files#copy
  2. StandardCopyOption#COPY_ATTRIBUTES
  3. Files#walkFileTree
  4. Apache Commons IO#FileUtils

List all files from a directory in Java

In this post, we will learn how to list all files from a directory in Java. We will be covering following techniques.

  1. Use Java8 Stream API.
  2. Using NIO Walk File Tree.
  3. File Walker.
  4. Apache Commons IO

At the end of this post, we will get a deep understanding of the performance for each of these methods and will discuss the best option to list all files recursively from a directory (subdirectories) in Java.

 

1. Using Java8 Steam API

public class ListFilesByStream {

    public static void main(String[] args) throws IOException {

        Path source = Paths.get("/Users/umesh/personal/tutorials/source");
        Files.walk(source).filter(Files::isRegularFile).forEach(System.out::println);
    }
}

Above method will return a Stream that is lazily populated with Path by walking the file tree rooted at a given starting file.

One of the main advantages of this method is Stream which provides flexibility to apply all different operations on returned stream (e.g. limit, grouping etc.).

 

If you want to list all files based on File Attributes or want to filter it, use Files.find() method by passing  BiPredicate.

 

2. Using FileVisitor walkFileTree

FileVisitor walkFileTree provides another convent way to recursively visit all the files in a file tree. We can use it to walk through the tree to perform any operation

  1. Read Files.
  2. Delete Files
  3. Move Files 

Please read  Deleting Directory in Java and Copy a File or Directory in Java to understand how to use FileVisitor to list all files from a directory in Java.

 

3. Using Apache Commons

public class ListFilesByApache {

    public static void main(String[] args) throws IOException {

        File source = new File("/Users/umesh/personal/tutorials/source");
        List<File> files = (List<File>) FileUtils.listFiles(source, TrueFileFilter.INSTANCE, TrueFileFilter.INSTANCE);
        for (File file : files) {
            // work on the files
        }
    }
}

You can also use iterateFiles to iterate in the source directory. While using Apache Commons File Utils, keep in mind following points.

  1. You can use fileFilter and dirFilter to filter files and directories.
  2. Subdirectories will not be included in the search by default, We have used TrueFileFilter.INSTANCE to include all files and subdirectories.
  3. Use FileFilterUtils to filter specific files/ directories from the search.

 

Performance

Benchmarking tests of recursively iterating through the files and directories indicate that Java8 Stream API is the fastest and Apache Commons IO was the slowest.

In case you are ready using Java8 (Which you should), use Java8 Stream API, else use Java7 NIO Walk tree method.

 

All the code of this article is available Over on Github. This is a Maven-based project.

References

  1. FileUtils
  2. Files#walk
  3. Files#find
  4. BiPredicate
  5. FileFilterUtils

Deleting Directory in Java

Deleting Directory in Java

It is possible to delete an empty directory. In this post, we will learn Deleting Directory in Java with contents. Deleting entire directory or directories with content is not simple in Java. Java 7  came up with easy and elegant solutions to address such requirements.

We will be covering following options

  1. Delete directories in Java using Java 7  FileVisitor.
  2. Apache Commons FileUtils.

 

1. Delete Directory Using Java

If we run following Java program assuming that root location is not empty (there are more directories inside tutorials directory)

 


public class DeleteDirectory {

  public static void main(String[] args) throws IOException {
        Path path = Paths.get("/Users/personal/tutorials");
        Files.delete(path);
    }
}

We will get following exception


Exception in thread "main" java.nio.file.DirectoryNotEmptyException: /Users/personal/tutorials
	at sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:242)
	at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
	at java.nio.file.Files.delete(Files.java:1126)
	at com.umeshawasthi.tutorials.corejava.io.directory.delete.DeleteDirectory.main(DeleteDirectory.java:15)

 

1.1 Delete Directory Using Java 7

With Java 7+ We can use convenient Files class

public class DeleteDirectory {

    public static void main(String[] args) throws IOException {
        Path rootDirectory = Paths.get("/Users/personal/tutorials/Write");

        Files.walkFileTree(rootDirectory, new SimpleFileVisitor() {

            @Override
            public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
                Files.delete(file);
                return FileVisitResult.CONTINUE;
            }

            @Override
            public FileVisitResult postVisitDirectory(Path dir, IOException exc) throws IOException {
                Files.delete(dir);
                return FileVisitResult.CONTINUE;
            }
        });
    }
}

 

1.2 Delete Directory Using Java NIO and Stream API

Java NIO and Stream API provides elegant options of deleting the directory in Java recursively.


public class DeleteDirectoryByStream {

    public static void main(String[] args) throws IOException {

        Path rootDirectory = Paths.get("/Users/personal/tutorials/stream");
        Files.walk(rootDirectory)
                .sorted(Comparator.reverseOrder())
                .map(Path::toFile)
                .forEach(File::delete);
     }
}

 

Here are few details for the above code

  1. Files.walk will return all files/directories under rootDirectory.
  2. We are using Comparator.reverseOrder() to ensure the main directory will be picked in the last.
  3. map Returns a File object representing this path.
  4. forEach will call delete method for each File object.

 

Above code block will not delete symlinks.To follow and delete symlinks use FileVisitOption.FOLLOW_LINKS as an additional parameter. Files.walk(rootDirectory, FileVisitOption.FOLLOW_LINKS)

 

1.1 Delete Directory Using Apache Commons

 

If your project is using Apache Commons, FileUtils provides an easier one line solution for Deleting Directory in Java.

 

public class DeleteDirectoryByApache {

    public static void main(String[] args) throws IOException {

        FileUtils.deleteDirectory(new File("/Users/personal/tutorials/apachecommons"));
    }
}

The code will not throw any exception in case path/ location is not correct.

Include Apache commons in your project if you want to use Apache Commons.

<dependency>
   <groupId>commons-io</groupId>
   <artifactId>commons-io</artifactId>
   <version>2.5</version>
</dependency>

 

If you want to create directory using Java NIO, read How to create directory in Java

 

All the code of this article is available Over on Github. This is a Maven-based project.

References

  1. FileVisitResult
  2. Files#walk
  3. FileUtils#deleteDirectory
  4. FileVisitOption#FOLLOW_LINKS

How to write to file in Java

In this post, we’ll learn different options as How to write to file in Java. We will be covering following options.

  1. Java 7 Files API
  2. BufferWriter.
  3. Apache Commons FileUtils.

 

1.  Java 7 Files API

Java 7  introduced the enhancement to work with the filesystem.Files class provide utility methods to create, delete or move files.

 

public class WriteFile {

    public static void main(String[] args) throws IOException {

        String fileContent = "We will demo as how to write using Java 7 File API";

        Path path = Paths.get("/tutorials/filewrite/testFile.txt");
        byte[] fileByteContent = fileContent.getBytes();

        Files.write(path, fileByteContent);

        //to validate our data
        String readFile = Files.readAllLines(path).get(0); //we will just read first line
        System.out.println(readFile.equals(fileContent));
    }
}

 

Above programme can be combined with one line

Files.write(Paths.get("/tutorials/filewrite/testFile.txt"), fileContent.getBytes());

Keep in mind following points while using Files Class

  • getBytes() method use default charset (UTF-8), it’s always recommended to pass charset to avoid any surprises.

 

2.  Using BufferWriter

 

BufferWriter provides the simplified way to write to file in Java. We will check following options to while using BufferWriter

  1. Use BufferWriter with FileOutputStream.
  2. Use BufferWriter with Files Class

In both examples, we will be using Java Try With Resources feature for automatic resource management.

 


public class WriteFileByBufferedWriter {

    public static void main(String[] args) throws IOException {

        String fileContent = "We will demo as how to write using Java BufferWriter";
        String fileLocation = "tutorials/filewrite/";

        writeFileByOutputStream(fileContent, fileLocation + "file1.txt");
        writeFileUsingFilesBufferedWriter(fileContent, fileLocation + "file2.txt");
    }


    public static void writeFileByOutputStream(final String fileContent, final String fileLocation) throws IOException {

        try (BufferedWriter out = new BufferedWriter(new OutputStreamWriter(
                new FileOutputStream(fileLocation), "UTF-8"))) {
            out.write(fileContent);
        }

    }

    public static void writeFileUsingFilesBufferedWriter(final String fileContent, final String fileLocation) throws IOException {
        Path filePath = Paths.get(fileLocation);
        try (BufferedWriter writer = Files.newBufferedWriter(filePath, StandardCharsets.UTF_8)) {
            writer.write(fileContent);
        }
    }

 

3.  Append String to Existing File

Buffer writer can be used to append data to an existing file.You need to create BufferReader in Append Mode.

 

newBufferedWriter(Path path, Charset cs, OpenOption... options),

The options parameter specifies how the file is created or opened. In above example, we created BufferWriter in Append Mode.


public class BufferWriterAppend {

    public static void main(String[] args) throws IOException {

        String fileContent = "This is a demo to show BufferWriter";
        String fileContent1 = " In Append Mode";
        String fileLocation = "/Users/umesh/personal/tutorials/filewrite/sampleFile.txt";

        Path filePath = Paths.get(fileLocation);
        try (BufferedWriter writer = Files.newBufferedWriter(filePath, StandardCharsets.UTF_8, StandardOpenOption.APPEND)) {
            writer.append(fileContent);
            writer.append(fileContent1);
        }
    }
}

Above code assume that you already have sampleFile.txt in the given location else system will throw NoSuchFileException

4. Using Apache Commons IO 

public class FileByApacheUtils {

    public static void main(String[] args) throws IOException {

        String fileContent = "We will demo as how to write using Java Apache Commons IO Utils";
        FileUtils.write(new File("/tutorials/filewrite/apacheIO.txt"),fileContent, StandardCharsets.UTF_8);
     }
}

In case you need to write a large file, use FileUtils.writeLines() method for the line by line.

We saw some of the most common and popular methods outlining How to write to file in Java. Additional options to write to file in Java.

  1. PrintWriter. (For writing formatted text)
  2. FileOutputStream (Mostly for writing binary data to file)
  3. DataOutputStream.
  4. RandomAccessFile (Ability to write file at a specific position)
  5. FileChannel

For creating directory read our post on  How to create a directory in Java.

 

All the code of this article is available Over on Github. This is a Maven-based project.

References

How to create directory in Java

How to create directory in Java

In this post, we will see how to create directory in Java.

We will be covering following options to understand how to create directory in Java

  1. Create a directory using file I/O introduced in Java 7 (java.nio package).
  2.  Apache Commons IO

 

1. Using Java 7

Java 7 Introduced java.nio.file system which provided 2 methods for creating a directory in Java.

public static Path createDirectory(Path dir, FileAttribute<?>... attrs)

public static Path createDirectories(Path dir,FileAttribute<?>... attrs)


createDirectory method is used to create a directory but will throw an exception in case parent directory does not exist into the system. Use this method only when you parent directory already exists in the file system.

createDirectories method is useful when we want to create all nonexistence nonexistent parent directories first.

1.1 Use CreteDirectory Method

 

public class CreateDirectory {

    public static void main(String[] args) throws IOException {

        Path path = Paths.get("/Users/personal/tutorials/directory");
        Path newPath= Files.createDirectory(path);
    }
}

Above code will create new directory with name "directory" under tutorials.

Above code will throw NoSuchFileException in case hierarchy does not exist.If we will change hierarchy to /Users/personal/tutorials1/directory (note tutorials1 does not exist in the underlying file system)

Output


Exception in thread "main" java.nio.file.NoSuchFileException: /Users/personal/tutorials1/directory
	at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
	at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:384)
	at java.nio.file.Files.createDirectory(Files.java:674)
	at com.umeshawasthi.tutorials.corejava.io.directory.CreateDirectory.main(CreateDirectory.java:16)

It will throw FileAlreadyExistsException in the case of directory structure already exists.Note that for createDirectory, check for the existence of the file and the creation of the directory if it does not exist are a single operation

1.2 Use CreteDirectories Method

For creating all nonexistent parent directories first, createDirectories method should be used

 


public class CreateDirectories {

    public static void main(String[] args) throws IOException {
        Path path = Paths.get("/Users/umesh/personal/tutorials_new/newDirectory");
        Files.createDirectories(path);
    }
} 

The method will not throw an exception if the directory could not be created because it already exists.In case it fails, it may do so after creating some, but not all, of the parent directories.

Both these methods will delegate work to FileSystemProvider.

FileAttribute attrs... is an optional parameter which can be used to set file attributes while creating file /directory.

For most of the case, FileAttribute will be set automatically.

 

1. Using Apache Commons IO

Apache Commons FileUtils#forceMkdir provide a convenient and easy way to create a directory in Java.


public class CreateDirectoryByFileUtils {

    public static void main(String[] args) throws IOException {
        FileUtils.forceMkdir(new java.io.File("/Users/umesh/personal/tutorials/directory"));
    }
}

FileUtils class provides two methods to create directory structure

public static void forceMkdir(File directory)
public static void forceMkdirParent(File file)

In this post, we checked different options how to create a directory in Java. Java 7 NIO package got a good enhancement to the file I/O package.

All the code of this article is available Over on Github. This is a Maven-based project.

References

  1. CreateDirectory
  2. CreateDirectories
  3. FileUtils.html#forceMkdir
  4. FileUtils#forceMkdirParent

Java String to Inputstream

Java String to Inputstream

In this post, we will go through different options at how to convert  Java String to InputStream using Java and Apache Commons IO library

1. Convert Using Plain Java

Use ByteArrayInputStream to convert Java String to InputStream

 

public class StringToIOJava {

    public static void main(String[] args) throws IOException {

        String inputString = "This is a String to demo as how to convert it to input stream using Core Java API";

        //String to input stream
        try (InputStream inputStream = new ByteArrayInputStream(inputString.getBytes(StandardCharsets.UTF_8));
             BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(inputStream))
        ) {
            String content;

            while ((content = bufferedReader.readLine()) != null) {
                //work with file
            }

        }
    }
}

 

 It’s always recommended to pass explicit Charset to the method getBytes() to avoid any unwanted behaviour.Java will pick platform’s default charset if not passed to the method.

 

1. Convert Using Apache Commons IO

Apache Commons IO provides short and concise solution for convert String to InputStream

 

public class StringToIOApache {

    public static void main(String[] args) throws IOException {

        String inputString = "This is a String to demo as how to convert it to input stream using Apache Commons IO";

        InputStream inputStream = IOUtils.toInputStream(inputString, StandardCharsets.UTF_8);

    }
}

In this post, we saw two easy and concise way to convert simple String to InputStream.

Please read Convert InputStream to String in Java to check how to Convert InputStream to String in Java.

 

All the code of this article is available Over on Github. This is a Maven-based project.

References

  1. ByteArrayInputStream
  2. Apache Commons IO

Convert Stack Trace to String in Java

Convert Stack Trace to String in Java

Handling exceptions in Java require an exception to storing in a file or might need it for some additional analysis than simply logging it. In this post, we will cover how to convert stack trace to string in Java.

1 Using Core Java API

Core Java API provides easy and efficient way to convert stack trace to string using StringWriter and PrintWriter

 

 
StringWriter writer = new StringWriter();
PrintWriter printWriter= new PrintWriter(writer);
exception.printStackTrace(printWriter);

 

 
public class StackTraceToStringJava {

    public static void main(String[] args) {

        try {
            throwDummyException();
        } catch (FileNotFoundException exception) {

            StringWriter writer = new StringWriter();
            PrintWriter printWriter= new PrintWriter(writer);
            exception.printStackTrace(printWriter);
            System.out.println("Exception in String is :: " + writer.toString());
        }
    }

    public static void throwDummyException() throws FileNotFoundException {
        throw new FileNotFoundException("Throwing file not found exception for demo purpose");
    }
}

 

Output

 
Exception in String is :: java.io.FileNotFoundException: Throwing file not found exception for demo purpose at com.umeshawasthi.tutorials.corejava.exception.StackTraceToStringJava.throwDummyException(StackTraceToStringJava.java:26)
at com.umeshawasthi.tutorials.corejava.exception.StackTraceToStringJava.main(StackTraceToStringJava.java:15)

Calling writer.toString() will provide stack trace in String format.

 

2 Using Apache Commons Lang API

Apache Commons Lang API ExceptionUtils class can be used to convert stack trace to String with a simple one line.

 


public class StackTraceToStringApache {

    public static void main(String[] args) {

        try {
            throwDummyException();
        } catch (FileNotFoundException exception) {
            String stackTraceAsString= ExceptionUtils.getStackTrace(exception);
            System.out.println("Strack Trace using Apache Commons :" + stackTraceAsString);
        }
    }

    public static void throwDummyException() throws FileNotFoundException {
        throw new FileNotFoundException("Throwing file not found exception for demo purpose Apache Commons Lang");
    }
}

Output


Strack Trace using Apache Commons :java.io.FileNotFoundException: Throwing file not found exception for demo purpose Apache Commons Lang
	at com.umeshawasthi.tutorials.corejava.exception.StackTraceToStringApache.throwDummyException(StackTraceToStringApache.java:23)
	at com.umeshawasthi.tutorials.corejava.exception.StackTraceToStringApache.main(StackTraceToStringApache.java:15)

To Apache Commons Lang API, we need to add it to classpath, We will use Maven for this (Simply add to lib folder if not using Maven)

<dependency>
  <groupId>org.apache.commons</groupId>
  <artifactId>commons-lang3</artifactId>
  <version>3.5</version>
</dependency>

3 Using Guava’s Throwable Class

Google Guava class provide Throwable utility class to convert stack trace to String

 String stackTrace = Throwables.getStackTraceAsString ( exception ) ;

We saw there are a number of different ways to Convert stack trace to String in Java using Java core API and third party API available.

Java 9 will be introducing StackWalker API which will provide more easy and efficient way to perform these operations.

 

All the code of this article is available Over on Github. This is a Maven-based project.

References

  1. Apache Commons Lang
  2. Guava Class Throwables
  3. Java 9 StackWalker

Java try with resources

Java try with resources

In Java, we open a file in a try block and close it in finally block to avoid any potential memory leak. Java try with resources was introduced in Java 7.

This new feature of Java try with resources statement ensure that resources will be closed after execution of the program. Resources declared under try with java resources must implement java.lang.AutoCloseable interface.

Here are few of the advantages of using Java try with resources statement.

  1. You don’t have to close resource explicitly, close() is automatically called by JVM, if it throws an IOException, it will be suppressed. (Call it automatic resource management)
  2. More clean and concise code.
  3. Ability to open multiple resources and manage them automatically.

 

Before Java 7, We will handle resource in following way


try{
      // Open resource (e.g FileStream)
    }
    catch(IOException exception){
        //exception handling
    }
    finally{
        //block to close resoucres opened in try block
    }

Java 7  try with resources introduced a more easy way to handle it.


try(//open your resource(s) here ){
        // Open resource (e.g FileStream)
    }
    // no need of finally, JVM will handle it for us

 

Pay close attention to try(//open your resource(s) here ) which is the new and main block for Java try with resource feature.

1. Resource Handling Pre Java 7

public class FIleIOPreJDK7 {

    public static void main(String[] args) throws FileNotFoundException {

        String fileName = "/tutorials/fileread/SampleFile.txt";
        String fileContent;
        BufferedReader bufferedReader = new BufferedReader(new FileReader(fileName));
        try {

            while ((fileContent = bufferedReader.readLine()) != null) {
                //business logic for file processing
            }
        } catch (IOException e) {
            e.printStackTrace();
        } finally {
            if (bufferedReader != null) {
                try {
                    bufferedReader.close();   // we need to close it to avoid memory leak.
                } catch (IOException ioException) {
                    ioException.printStackTrace();
                }
            }
        }

    }
}

2. With Java 7

public class FileIOWithJava7 {

    public static void main(String[] args) {

        String fileName = "/tutorials/fileread/SampleFile.txt";

        try( BufferedReader bufferedReader = new BufferedReader(new FileReader(fileName))){
            String content;
            while((content = bufferedReader.readLine() )!=null){
                //you logic here
            }
        }
        catch (IOException ioException) {
            ioException.printStackTrace();
        }
    }
}

In the above example,  the resource declared in the try-with-resources statement is a BufferedReader.Resource declaration appears within parentheses immediately after the try keyword.

You must have noticed that there is no finally in the code, try with resource statement will automatically close BufferReader once program complete try block.

In this section, we will cover other features of  Java try with resources.

3. Try with resources for multiple Resources

try with resources can efficiently be used for multiple resources

 

public class MultipleResouces {

    public static void main(String[] args) {

        String fileName = "/tutorials/fileread/SampleFile.txt";
        String fileName1 = "/tutorials/fileread/SampleFile1.txt";

        try (BufferedReader bufferedReader = new BufferedReader(new FileReader(fileName));
             Scanner scanner = new Scanner(new File(fileName1))) {
            String content;
            while ((content = bufferedReader.readLine()) != null) {
                //you logic here
            }
        } catch (IOException ioException) {
            ioException.printStackTrace();
        }
    }
}

Multiple resources can be declared within try block by using

; (try (BufferedReader bufferedReader = new BufferedReader(new FileReader(fileName));
             Scanner scanner = new Scanner(new File(fileName1))))

4. Custom Resource

To create custom resource, you need to implement AutoCloseable interface

MyCustomResource impliments AutoCloseable{
  @Override 
  public void close() throws Exception{
     //you need to impliment close method
  }
}

5. Improvement with JDK9

As per of the Milling Project Coin in JDK 9, there is further improvement in the Java try with resources statement which is more natural” way of writing.

If you already have resource as a final or a final variable, you can easily use this variable in try statement without declaring any a new variable

Before JDK9

// BufferedReader is declared outside try() block
BufferedReader br = new BufferedReader(new FileReader("file-path"));

try (BufferedReader inBr = br) {
    // ...
} catch (IOException e) {
    // ...
}

With JDK9

// JDK9
BufferedReader br = new BufferedReader(new FileReader("file-path"));

try (br) {
    // use the reader
} catch (IOException e) {
    // ignoring exceptions because that is how I roll
}

All the code of this article is available Over on Github. This is a Maven-based project.

Reference

  1. try-with-resources
  2. AutoCloseable
  3. Closeable

How to Read Large File in Java

How to Read Large File in Java

In our last article, we cover How to read file in Java.This post will cover how to read large file in Java efficiently.

Reading the large file in Java efficiently is always a challenge, with new enhancements coming to Java IO package, it is becoming more and more efficient.

We have used sample file with size 1GB for all these. Reading such a large file in memory is not a good option, we will covering various methods outlining How to read large file in Java line by line.

 

1 Using Java API

We will cover various options how to read file in Java efficiently using plain Java API.

 

 1.1 Using Java BufferReader


public class ReadLargeFileByBufferReader {

    public static void main(String[] args) throws IOException {

        String fileName = "/tutorials/fileread/file.txt"; //this path is on my local
        try (BufferedReader fileBufferReader = new BufferedReader(new FileReader(fileName))) {
            String fileLineContent;
            while ((fileLineContent = fileBufferReader.readLine()) != null) {
                // process the line.
            }
        }
    }
}

Output


Max Memory Used : 258MB
Time Take : 100 Seconds

 1.2 Using Java 8 Stream API

public class ReadLargeFIleUsingStream {

    public static void main(String[] args) throws IOException {

        String fileName = "/tutorials/fileread/file.txt"; //this path is on my local
        // lines(Path path, Charset cs)
        try (Stream inputStream = Files.lines(Paths.get(fileName), StandardCharsets.UTF8)) {
            inputStream.forEach(System.out::println);
        }
    }
}

 

Output

Max Memory Used : 390MB
Time Take : 60 Seconds


 1.3 Using Java Scanner

Java Scanner API also provides a way to read large file line by line.


  public class ReadLargeFileByScanner {

    public static void main(String[] args) throws FileNotFoundException {
        String fileName = "/Users/umesh/personal/tutorials/fileread/file.txt"; //this path is on my local
        InputStream inputStream = new FileInputStream(fileName);
       
        try(Scanner fileScanner = new Scanner(inputStream, StandardCharsets.UTF_8.name())){
            while (fileScanner.hasNextLine()){
                System.out.println(fileScanner.nextLine());
            }
        }
    }
}

Output


Max Memory Used : 460MB
Time Take : 60 Seconds

2 Streaming File Using Apache Commons IO

This can also be achieved by using Apache Commons IO  FileUtils.lineIterator () Method


public class ReadLargeFileUsingApacheCommonIO {

    public static void main(String[] args) throws IOException {
        String fileName = "/Users/umesh/personal/tutorials/fileread/file.txt"; //this path is on my local

        LineIterator fileContents= FileUtils.lineIterator(new File(fileName), StandardCharsets.UTF_8.name());
        while(fileContents.hasNext()){
            System.out.println(fileContents.nextLine());   
        }
    }
}

Output


Max Memory Used : 400MB
Time Take : 60 Seconds

As we saw how to read large file in Java efficiently. Few things which you need to pay close attention

  1. Reading the large file in one go will not be a good option (You will get OutOfMemoryError ).
  2. We Adapted technique to read large file line by line to keep memory footprint low.

I used VisualVM to monitoring Memory, CPU and Threadpool information while running these programmes.

based on our test, BufferReader has the lowest memory footprint, though the overall execution was slow.

All the code of this article is available Over on Github. This is a Maven-based project.

References

  1. Apache Commons IO

How to read file in Java

How to read file in Java

In this post, We will cover as  How to read file in Java. This post will cover different options available in Java to see how to read file in Java.

1.  Reading File Using JDK

JDK provide number of option to read file in Java, we will explore few of the options to read file

  1.1 Read File using Java 8 Stream API

Java 8 Provides an efficient way to read a file using Stream API. You can use Files.line to read it as a  stream

File Content

This Show How
To Read File
Using Java 8
public class ReadFileByStream {

    public static void main(String[] args) {

       String fileName = "/tutorials/fileread/SampleFile.txt"; //this path is on my local
        try {
            List<String> readStream=Files.lines(Paths.get(fileName)).collect(Collectors.toList());
            readStream.forEach(System.out::println);
        } 
catch (IOException e) {
            e.printStackTrace();
        }
    }
}

Output

This Show How
To Read File
Using Java 8

Java 8 Stream API is really clean and provide an elegant way to read the file.We have seen too many open files in a system exception being reported.

 

1.2 Read File using BufferReader and StringBuilder

BufferReader with StringBuilder is an efficient way to read small file in Java

public class ReadFileByBufferReader {

    public static void main(String[] args){

        String fileName = "/tutorials/fileread/SampleFile.txt";

        try(BufferedReader bufferReader = new BufferedReader(new FileReader(fileName))) {
            StringBuilder fileContent = new StringBuilder();
            String line = bufferReader.readLine();

            while (line != null) {
                fileContent.append(line);
                fileContent.append(System.lineSeparator());
                line = bufferReader.readLine();
            }

            String fileInformation = fileContent.toString();
            System.out.println(fileInformation);

        } catch (FileNotFoundException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

Output

This Show How
To Read File
Using Buffer Reader

Above Example used try-with-resources feature to autoclose stream.

1.3 Read File using Scanner

package com.umeshawasthi.tutorials.corejava.io.readfile;

import java.io.File;
import java.io.FileNotFoundException;
import java.util.Scanner;

/**
 * Created by umesh on 5/2/17.
 */
public class ReadFileByScanner {

    public static void main(String[] args) throws FileNotFoundException {

        String fileName = "/tutorials/fileread/SampleFile.txt";
        File inputFile = new File(fileName);

        useScannerByLoop(inputFile);
        useScannerWithoutLoop(inputFile);

    }

    private static void useScannerByLoop(final File file) throws FileNotFoundException {
        Scanner fileScanner = new Scanner(file);
        while(fileScanner.hasNextLine()){
            System.out.println(fileScanner.nextLine());
        }
        fileScanner.close();
    }

    private static void useScannerWithoutLoop(final File file) throws FileNotFoundException {
        Scanner fileScanner = new Scanner(file);
        fileScanner.useDelimiter("\\Z");
        System.out.println(fileScanner.next());

    }
}

Output (For both cases)

This Show How
To Read File
Using Scanner
With Loop

Make sure to check next line when using Scanner else it can throw NoSuchElementException exception

 

1.4  Apache Commons IO Utils

Apache Commons provide the convenient method to work with files.

Here is a clean and efficient way to as how to read file in Java using Apache Commons IO Utils.

public class ReadFileByApacheIO {

    public static void main(String[] args) throws IOException {

        String fileName = "/tutorials/fileread/SampleFile.txt";
        try(FileInputStream inputStream = new FileInputStream(fileName)){

            String fileContent = IOUtils.toString(inputStream, StandardCharsets.UTF_8);
            System.out.println(fileContent);
        }
    }
}

Output

This Show How
To Read File
Using Apache
IO

We have used Apache Commons IOUtils using Java 7  try-with-resources feature.

Apache FileUtils

public class ReadFileByApacheFileUtils {

    public static void main(String[] args) throws IOException {

        String fileName = "/Users/umesh/personal/tutorials/fileread/SampleFile.txt";
        List<String> fileContent= FileUtils.readLines(new File(fileName), StandardCharsets.UTF_8);

        for(String line : fileContent){
            System.out.println(line);
        }
    }
}

Output

This Show How
To Read File
Using Apache
IO

There are multiple ways as How to Read file in Java. All options have their own positive and negative.

Our recommendation is

  1. Apache Commons
  2. Buffer Reader
  3. Others

If you are using Google Guava, you can use Guava in a very similar way Apache Commons IO.

Please refer to our post on how to convert InputStream to String for a basic understanding of the process.

All the code of this article is available Over on Github. This is a Maven-based project.

References

  1. Apache Commons IOUtils
  2. Apache Commons IO FileUtils