EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  <20122013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  <20122013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
<== Date ==> <== Thread ==>

Subject: 32-bit TIFFs and PIL
From: Mark Rivers <[email protected]>
To: Timothy Madden <[email protected]>
Cc: "[email protected]" <[email protected]>, Ross Harder <[email protected]>, "[email protected]" <[email protected]>
Date: Tue, 20 Mar 2012 13:03:00 +0000
Hi Tim (and tech-talkers),

I know you've done quite a bit with TIFF files, so I've got a question for you.

The areaDetector NDFileTIFF plugin can write 8-bit, 16-bit and 32-bit TIFF files.  All of them can be read fine by ImageJ and IDL.  However, the 32-bit TIFFs cannot be read by the Python Imaging Library (PIL).  But 32-bit TIFF files written by the Dectris Pilatus driver can be read by PIL.

Do you have any idea what needs to be done to make a 32-bit TIFF readable by PIL?

Thanks,
Mark

________________________________________
From: [email protected] [[email protected]] on behalf of Matt Newville [[email protected]]
Sent: Monday, March 19, 2012 10:34 PM
To: Mark Rivers
Cc: Ross Harder; [email protected]
Subject: Re: tiffs

Hi Mark,

Oh, right, I was using tiffinfo on my Mac, after moving the files over
(so that the display was faster).   tiffinfo -v says 3.9.4  I
installed libtiff-tools on corvette, so tiffinfo (3.9.5) now works
there too.

---Matt

On Mon, Mar 19, 2012 at 10:19 PM, Mark Rivers <[email protected]> wrote:
> Hi Matt,
>
> Thanks for that information.
>
> What version of tiffinfo are you running?  Your output format is slightly different from what I get on idea and millenia.  I'd like to run tiffinfo on a Pilatus file and on the areaDetector file and see how they differ.  Then maybe I can change areaDetector's writer slightly to produce files that PIL can read.
>
> Mark
>
> ________________________________________
> From: [email protected] [[email protected]] on behalf of Matt Newville [[email protected]]
> Sent: Monday, March 19, 2012 9:54 PM
> To: Mark Rivers
> Cc: Ross Harder; [email protected]
> Subject: Re: tiffs
>
> Hi Mark, Ross, David,
>
> On Mon, Mar 19, 2012 at 5:46 PM, Mark Rivers <[email protected]> wrote:
>> Ross,
>>
>> I just talked to Matt Newville.  He thought that maybe this was a problem with an older version of PIL.  However, I just tried it on our new Linux system (corvette) and I can read 8-bit and 16-bit TIFFs fine in PIL, but 32-bit does not work.
>>
>> I used the simDetector to create 8-bit, 16-bit and 32-bit integer TIFF files.
>>
>> I then read them in IDL.  It worked fine.
>>
>> IDL> t = read_tiff('8bit_int_1.tiff')
>> IDL> help, t
>> T               BYTE      = Array[1024, 1024]
>> IDL> t = read_tiff('16bit_int_1.tiff')
>> IDL> help, t
>> T               UINT      = Array[1024, 1024]
>> IDL> t = read_tiff('32bit_int_1.tiff')
>> IDL> help, t
>> T               ULONG     = Array[1024, 1024]
>>
>> I then tried to read them in PIL.  8-bit and 16-bit worked, but not 32-bit.
>>
>> corvette:~/scratch>python
>> Python 2.7.1 (r271:86832, Apr 12 2011, 16:15:16)
>> [GCC 4.6.0 20110331 (Red Hat 4.6.0-2)] on linux2
>> Type "help", "copyright", "credits" or "license" for more information.
>> Imported history from: '/home/epics/.pyhist'
>>>>> from PIL import Image
>>>>> im = Image.open("8bit_int_1.tiff")
>>>>> im.show()
>>>>> im = Image.open("16bit_int_1.tiff")
>>>>> im.show()
>>>>> im = Image.open("32bit_int_1.tiff")
>> Traceback (most recent call last):
>>  File "<stdin>", line 1, in <module>
>>  File "/usr/lib64/python2.7/site-packages/PIL/Image.py", line 1980, in open
>>    raise IOError("cannot identify image file")
>> IOError: cannot identify image file
>>
>> Matt, is the version of PIL I am running on corvette with Python 2.7.1 a recent version?  Do you understand why I can't read 32-bit TIFFs?
>>
>> Mark
>>
>>
>> -----Original Message-----
>> From: Mark Rivers
>> Sent: Monday, March 19, 2012 2:59 PM
>> To: 'Ross Harder'; '[email protected]'
>> Subject: RE: tiffs
>>
>> Hi Ross,
>>
>> I just ran the Linux utility "tiffinfo" on your file:
>>
>> idea:~>tiffinfo /corvette/home/epics/scratch/im.tiff
>> TIFF Directory at offset 0x3fe00c (4186124)
>>  Image Width: 1023 Image Length: 1023
>>  Bits/Sample: 32
>>  Compression Scheme: None
>>  Photometric Interpretation: min-is-black
>>  Samples/Pixel: 1
>>  Rows/Strip: 1023
>>  Planar Configuration: single image plane
>>  Make: Unknown
>>  Model: Unknown
>>
>> It also says that the image directory is near the end of the file.  But it is clearly a valid TIFF file, so why doesn't PIL like it?
>>
>> Mark
>>
>>
>> -----Original Message-----
>> From: Mark Rivers
>> Sent: Monday, March 19, 2012 2:55 PM
>> To: 'Ross Harder'; [email protected]
>> Subject: RE: tiffs
>>
>> Hi Ross,
>>
>> It is not the "first image" that is near the end of the file, it is the "first image directory".  It looks like the image directory is being placed after the image.
>>
>> If you look at the NDFileTIFF plugin, you will see that it is very simple.  It uses libtiff.  For a 32-bit mono image these are the only calls to libtiff in the plugin, and they are done in this order:
>>
>>    TIFFOpen(fileName, "w")
>>    TIFFSetField(this->output, TIFFTAG_BITSPERSAMPLE, bitsPerSample);
>>    TIFFSetField(this->output, TIFFTAG_SAMPLESPERPIXEL, samplesPerPixel);
>>    TIFFSetField(this->output, TIFFTAG_PHOTOMETRIC, photoMetric);
>>    TIFFSetField(this->output, TIFFTAG_PLANARCONFIG, planarConfig);
>>    TIFFSetField(this->output, TIFFTAG_IMAGEWIDTH, sizeX);
>>    TIFFSetField(this->output, TIFFTAG_IMAGELENGTH, sizeY);
>>    TIFFSetField(this->output, TIFFTAG_ROWSPERSTRIP, rowsPerStrip);
>>    TIFFSetField(this->output, TIFFTAG_MAKE, ManufacturerString);
>>    TIFFSetField(this->output, TIFFTAG_MODEL, ModelString);
>>
>>    stripSize = TIFFStripSize(this->output);
>>    TIFFGetField(this->output, TIFFTAG_IMAGELENGTH, &sizeY);
>>
>>    nwrite = TIFFWriteEncodedStrip(this->output, 0, pArray->pData, stripSize);
>>    TIFFClose(this->output);
>>
>> So if the directory is being put at the end of the file, it must be because that's how libtiff does it.
>>
>> Mark
>>
>>
>> -----Original Message-----
>> From: [email protected] [mailto:[email protected]] On Behalf Of Ross Harder
>> Sent: Monday, March 19, 2012 2:40 PM
>> To: Mark Rivers; [email protected]
>> Subject: tiffs
>>
>> Hi Mark,
>> I've placed a tiff from a recent scan on the xor ftp server.  I've
>> also included a short python program that interrogates the file.
>> I don't understand why the offset to the first image is almost at the
>> end of the file?
>>
>> http://ftp.xor.aps.anl.gov/sector33_34/34idc/tifread/
>>
>>
>> In [121]: run tiffinfo.py
>> Little endian is II: II
>> Sanity check for endianess should be 42 if a valid tif: (42,)
>> Offset of first image file directory in the tif: (4186124,)
>>
>> Any thoughts?
>>
>> Ross
>>
>
> Sorry for the trouble. I do remember that 32-bit images from a Pilatus
> could not be read with older versions of PIL, but can be read now
> (taking a random one from Nov 2011):
>
> ~> tiffinfo Pilatus_Nov2011.tif
>
> Pilatus_Nov2011.tif: Warning, incorrect count for field "DateTime"
> (32, expecting 20); tag trimmed.
> TIFFReadDirectory: Warning, Pilatus_Nov2011.tif: invalid TIFF
> directory; tags are not sorted in ascending order.
> TIFFReadDirectory: Warning, Pilatus_Nov2011.tif: unknown field with
> tag 36864 (0x9000) encountered.
> TIFFReadDirectory: Warning, Pilatus_Nov2011.tif: unknown field with
> tag 36865 (0x9001) encountered.
> TIFFReadDirectory: Warning, Pilatus_Nov2011.tif: unknown field with
> tag 36866 (0x9002) encountered.
> TIFFReadDirectory: Warning, Pilatus_Nov2011.tif: unknown field with
> tag 36867 (0x9003) encountered.
> TIFFReadDirectory: Warning, Pilatus_Nov2011.tif: unknown field with
> tag 36868 (0x9004) encountered.
> TIFFReadDirectory: Warning, Pilatus_Nov2011.tif: unknown field with
> tag 36870 (0x9006) encountered.
> TIFFReadDirectory: Warning, Pilatus_Nov2011.tif: unknown field with
> tag 36876 (0x900c) encountered.
> TIFFReadDirectory: Warning, Pilatus_Nov2011.tif: unknown field with
> tag 36877 (0x900d) encountered.
> TIFFReadDirectory: Warning, Pilatus_Nov2011.tif: unknown field with
> tag 36878 (0x900e) encountered.
> TIFFReadDirectory: Warning, Pilatus_Nov2011.tif: unknown field with
> tag 36879 (0x900f) encountered.
> TIFFReadDirectory: Warning, Pilatus_Nov2011.tif: unknown field with
> tag 36880 (0x9010) encountered.
> TIFFReadDirectory: Warning, Pilatus_Nov2011.tif: unknown field with
> tag 37120 (0x9100) encountered.
> Pilatus_Nov2011.tif: Warning, incorrect count for field "Tag 36864"
> (0, expecting 1); tag ignored.
> Pilatus_Nov2011.tif: Warning, incorrect count for field "Tag 36870"
> (0, expecting 1); tag ignored.
> TIFF Directory at offset 0x82 (130)
>  Subfile Type: (0 = 0x0)
>  Image Width: 487 Image Length: 195
>  Resolution: 69.5714, 69.5714
>  Bits/Sample: 32
>  Sample Format: signed integer
>  Compression Scheme: None
>  Photometric Interpretation: min-is-black
>  Rows/Strip: 195
>  Planar Configuration: single image plane
>  ImageDescription: # Pixel_size 172e-6 m x 172e-6 m
> # Silicon sensor, thickness 0.000320 m
> # Exposure_time 16.700000 s
> # Exposure_period 16.700000 s
> # Tau = 203.0e-09 s
> ....
>
> For this tiff file
>
>>>> import Image
>>>> Image.open('Pilatus_Nov2011.tif').show()
>
> displays fine.  That sure looks like a 32-bit integer TIFF to me,
> though tiffinfo wasn't all that happy with some of the tags.
>
> But I see exactly the same behavior for the tiff files that Mark
> created with simDetector, that Image.open() complains about not being
> able to identify the image file.  I'm not sure why that is....  The
> PIL docs are pretty opaque, but I believe this works:
>
>>>> data = open('32bit_int_1.tiff').read()   # raw file read
>>>> img  = Image.frombuffer('I', (1024, 1024), data, 'raw', 'I')
>>>> img.show()
>
> This shows one small bright spot in upper left, around pixel 200,100,
> with a max intensity of 186.  It's definitely lame that Image.open()
> doesn't auto-recognize this file!  I don't understand why it
> understood the Pilatus file, but not this one (all the
> Image.frombuffer() call is doing is datatype as an integer, and
> explicitly setting the size).
>
> BTW, to convert this to a numpy array, use
>>>> np_data = numpy.array(img.getdata(), dtype='int32').reshape(1024, 1024).transpose()
>
> Finally, if you're interested, I have a start of an Image Display GUI
> (wxPython) for Epics Area Detectors.  This uses exactly this
> 'Image.frombuffer()' method to convert data directly from an
> AreaDetector PV data.  Not much documentation yet, but code is at at
>  https://github.com/pyepics/epicsapps/tree/master/AreaDetector_Display
> and a screenshot is at
>  http://pyepics.github.com/epicsapps/ad_display.html
>
> I've also made a Windows EXE made from this, but haven't posted it online yet.
> Hope that helps,
>
> --Matt Newville <newville at cars.uchicago.edu> 630-252-0431
>



--
--Matt Newville <newville at cars.uchicago.edu> 630-252-0431


Replies:
Re: 32-bit TIFFs and PIL Matt Newville
Re: 32-bit TIFFs and PIL J. Lewis Muir

Navigate by Date:
Prev: BOY: Macro for workspace directory? Ralph Lange
Next: [ASYN-vxi11] can't connect to Tektronix scope on Fedora 16 Pavel Masloff
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  <20122013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
Navigate by Thread:
Prev: RE: Macro for workspace directory? Chen, Xihui
Next: Re: 32-bit TIFFs and PIL Matt Newville
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  <20122013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
ANJ, 18 Nov 2013 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·