I was under the impression that having a pre-computed Tfrecord file was the most efficient way to feed your input function. However, I keep seeing great looking articles such as this one where the input function takes a reference to a raw file on disk, and does the decoding on the spot.
- Is there a benefit in creating Tfrecord files, or is it just as efficient to decode and prepare each sample right inside the input function (as opposed to having the input function simply decode the Tfrecord with tensorflow)?
- When using direct raw files in the input function as in the example above, where would you add the data augmentation step?
The way I’ve done this in the past is that I’d have a separate script that, given a reference to some files, it would generate a Tfrecord file with the data augmentation as part of it. For example, the first n images in the Tfrecord were a given image, followed by random transformations of it, etc. Then the input function simply decoded each record and specified the batching, shuffling, etc.