Scaling images – Quirks and tricks

In one of our projects we created functionality to upload images. These images can, after being uploaded, be viewed on a grid or on  a detail page. The grid shows a thumbnail and the detail page shows a medium variant of the image. To create these variants we have used java ImageIO, imgscalr, ImageMagick (im4java) and Exiftool. Sounds like quite a few libraries to create two different sized images, but it’s the result of trying to support as many images as possible and provide a good user experience. In this blog post I will explain how and why we have used these libraries to solve the problems we encountered.

In our application we allow the user to upload the following file types: *.jp(e)g, *.png, *.tif(f).  For .png and .tiff files we have to do some extra steps before we can scale the image to a thumbnail and medium size variant. I will elaborate on that part later on. As you can see in the flow diagram, we store the full size image in the database. After that an event is started to create an asynchronous process. First lets take a look at how we handle .jp(e)g files.  We create a BufferedImage from our source file, which is a Path in our case:

BufferedImage bufferedImage =  ImageIO.read(source.toFile());

Well that was really easy, but with this one-liner we already discovered some problems during the process of testing some images. Not all images (no matter what content type, even if there is a reader for that content type) could be read by ImageIO.  For example, one of the images contained a color code that is not supported. Instead of trying to support all kind of scenarios and the risk of missing one, we decided added a fallback that we convert the image to jpg file with ImageMagick. Then the with converted image we try to read the image again with ImageIO.

private BufferedImage createBufferedImage(Path source)
throws IOException {
  try {
    // First try to create bufferedImage with original source
    return ImageIO.read(source.toFile());
  } catch (IOException e) {
    // If failed to create bufferedImage with original source,
    // try to create bufferedImage with converted jpg.
    // If that also fails, we can not handle the image and
    // the exception needs to be handled.
    Path imageWithoutExifMetadata =
            copyImageWithoutExifMetadataFromImage(source);
    Path convertedSource = convertToJpg(imageWithoutExifMetadata);
    BufferedImage bufferedImage =
            ImageIO.read(convertedSource.toFile());

    cleanupTemporaryConvertFiles(
            convertedSource, imageWithoutExifMetadata);

    return bufferedImage;
  }
}

Now that we have a BufferedImage we can use this to scale the image with imgscalr.

BufferedImage thumbnailed = Scalr.resize(bufferedImage,
        jpgImageQuality, Scalr.Mode.AUTOMATIC, newImageSize);

When the image is scaled we need to write the image:

ByteArrayOutputStream baos = new ByteArrayOutputStream();
ImageOutputStream ios = ImageIO.createImageOutputStream(baos);
Iterator<ImageWriter> iter = ImageIO.getImageWritersByFormatName("jpeg");
ImageWriter writer = iter.next();
ImageWriteParam iwp = writer.getDefaultWriteParam();
iwp.setCompressionMode(ImageWriteParam.MODE_EXPLICIT);
iwp.setCompressionQuality(jpgImageCompression);
if (iwp.canWriteProgressive() && originalFileSize > TEN_KB) {
  iwp.setProgressiveMode(ImageWriteParam.MODE_DEFAULT);
}
writer.setOutput(ios);
writer.write(null, new IIOImage(thumbnailed, null, null), iwp);
writer.dispose();

baos.flush();
byte[] imageInByte = baos.toByteArray();
baos.close();

As I said before we convert the image with ImageMagick in case we can not read it properly. Also here we added a few optimisations to convert the most images. First we have added the quiet mode option to ignore some of the warnings that made the conversion fail.

Sometimes images have multiple layers and ImageMagick will convert an image for each of those layers. Because we only need one image that we can use in our process, we always ask for the first layer. This can be done by adding [0] to the filename.

ConvertCmd cmd = new ConvertCmd();
cmd.setSearchPath(pathToImageMagick);

IMOperation op = new IMOperation();

// We addded the quiet mode to ignore some warnings which made the conversion failed
op.quiet();
op.colorspace("sRGB");
//Take the first layer ([0]) of the image to convert
op.addImage(imageFile.toAbsolutePath().toString() + "[0]");
op.addImage(imageFile.toAbsolutePath().toString() + JPG_EXTENSION);

cmd.run(op);

Tiff files

Tiff files can not be read by ImageIO, because there is no reader available. So each tiff file is converted to a jpg file using ImageMagick. This works fine, except for a few images we tried. These images failed to convert because of a bug in the underlying library libtif. After some heavy research I found that the conversion failed because the metadata could not be read properly.  The best way to solve this problem was to upgrade the libtif library (as suggested on the ImageMagick forum), but that was a bit of a problem on our operating system. Therefore we decided to remove the metadata with Exiftool from the tiff file and convert the image without the metadata. The converted image is then used for creating the thumbnail and medium variant.

private Path copyImageWithoutExifMetadataFromImage(Path imageFile) {
  Path tempImage;

  try {
    tempImage = Files.createFile(
            buildImageWithoutMetadataFilePath(imageFile));
    Files.copy(imageFile, tempImage,
            StandardCopyOption.REPLACE_EXISTING);

    metadataWriter.deleteExifMetadata(tempImage);
  } catch (IOException e) {
    logger.error("Failed to delete exif metadata from image " +
            imageFile.toAbsolutePath().toString(), e);
    throw new ImageScalingException(e);
  }
  return tempImage;
}

We now have a process in place that creates image variants for probably all our images. This gives us a solid base to upload the images and don’t bother the users with upload exceptions. Hopefully this process helps some of you in scaling, converting or creating images.