This seems to be more of an efficiency edit than a bug fix, correct? I'm failing to see how we would get an error from this write - it just removes some redundant writing that's occurring (please correct if I'm missing something). I think my recommendation would be to either:

a) move the write_bil_chunk sections one block up in the loop so that they only happen after the row loop (in which case you can revert to writing the whole chunk, not a section), to reduce disk throttling, or....
b) not hold all rows of output_rfl in memory

I would probably default to a).

I'll confirm once more that this is the root of the issue I was running into. It was clear that the numpy matrix writing into the output BIL files was not at the correct dimensions because the data structure in the resulting file was not oriented correctly.

@pgbrodrick Digging into the write statement in the AL:

The edited block

            write_bil_chunk(
                output_rfl.T,
                self.analytical_rfl_file,
                r,
                (rdn.shape[0], rdn.shape[1], len(self.fm.idx_surf_rfl)),
            )

attempts to write the entire output_rfl.T chunk at the r position within the file, which is a little strange. Further, output_rfl.T will be at shape rdn.shape[2], rdn.shape[1], rdn.shape[0], which is mismatched with the function call.

To your point, a change along the lines of a) makes sense to me as long as I'm understanding correctly. We could move the write_bil_chunk up to execute after both column and row level for loops. We are populating the output_rfl matrix regardless, which was previously going unused outside of the row-level indexing and writes.

I ran three EMIT test cases without this change and they all ran successfully. My guess is the error case I was hitting yesterday was due to an alteration I made. @pgbrodrick is correct on this.

With that said, keeping the output_rfl-type matrices in memory, but only using the row entries on the for loop is strange. Leaving this open for discussion.

Handled via merging #671