Mike Chaney's Tech Corner
November 24, 2024, 01:08:03 PM *
Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
News: Qimage registration expired? New lifetime licenses are only $59.99!
 
   Home   Help Login Register  
Mike Chaney's Tech Corner > Mike's Software > Qimage Ultimate > QI Ultimate very slow with 40 mp files
Pages: [1]
« previous next »
  Print  
Author Topic: QI Ultimate very slow with 40 mp files  (Read 16945 times)
aaturner
Newbie
*
Posts: 22


Email
« on: June 18, 2012, 03:39:12 PM »
Hi all. I have used QI Studio and Ultimate for some time now with great results. However, since upgrading my camera to a Nikon D800 enerything seems to have slowed down. Changing my RAW preferences from "Lightening Raw" to "Fast Thumbs" to use the embedded JPEG seems to have helped as the cacheing of raw thumbs was taking forever. Am I doing something wrong or am I just suffering from the difference between 12mp and 40mp?. My system is 32 bit (yes, I know, time to migrate to 64 bit and greater memory capacity) with a quad core 3 GHz processor. Any suggestions as to how to speed things up?; the actual printing process takes as long as it did before once my hdd has been ground to a pulp by endless cacheing, which is about as quick and exciting as watching grass grow! Many thanks for any suggestions.
Logged
Terry-M
The Honourable Metric Mann
Forum Superhero
*****
Posts: 3251



WWW
« Reply #1 on: June 18, 2012, 03:55:37 PM »
Hi,
Can you quote some actual times for caching so we can comment more meaningfully? What are the actual raw file sizes? I see the D800 has a 36.8 mpx sensor. How many images in a folder.
I'm probably going to upgrade my camera soon to one that produces18mpx/25 MB raw files. I downloaded some sample images and noted that they took 4.0 secs per image to make the raw cache and 0.8 secs per image to make the HQ thumbs
This on a 3.6GHZ quad core 64bit PC with 6GB of RAM.
On my machine, I would expect the cache time to be in proportion so around 7 sec per 36.8mpx image; I believe the processor speed has the biggest effect on caching speed.
Quote
Changing my RAW preferences from "Lightening Raw" to "Fast Thumbs"
This will switch off raw caching and make lower quality thumbs. I'm not convinced this is a good idea. Although the initial raw caching may take a little time, subsequent raw adjustments will take less time with caching on.
Terry
Edit: cache timing corrected and thumb timing added
« Last Edit: June 18, 2012, 07:50:20 PM by Terry-M » Logged
Fred A
Forum Superhero
*****
Posts: 5644



WWW Email
« Reply #2 on: June 18, 2012, 04:34:49 PM »
Quote
his will switch off raw caching and make lower quality thumbs. I'm not convinced this is a good idea. Although the initial raw caching may take a little time, subsequent raw adjustments will take less time with caching on.

To me, the main point is not so much how long it takes to cache a large file from a new camera, but that it only builds the cache once per folder.
After tha waiting period, you will be just as you were before. QU Opens and uses the Cached files which are virtually no loss JPGS while you work on them.
The only time a cache file is renewed is when you change something in the Raw file. Then it's one at a time anyway. So it becomes no bother.

Fred
Logged
Terry-M
The Honourable Metric Mann
Forum Superhero
*****
Posts: 3251



WWW
« Reply #3 on: June 18, 2012, 07:58:13 PM »
More cache speed data:
I downloaded some sample D800 images and times the thumbnail & raw cache speed., results as follows:
Thumbs 2.0 secs per image; Raw Cache 8.7 secs per image; Total 10.7 secs per image.
I also compared several raw image types and their speeds and found that in most cases the thumb-caching was  was directly proportional to the file size: on my PC it was ~4 seconds per MB total for thumbs and cache.
The one exception were files from a Canon 600D which were 25% faster for some reason.

Caching is demanding on the PC processor so I would recommend when upgrading to a new PC etc. to go for the fastest and most powerful you can afford. Probably cheap compared to the cost of a D800  Wink
It seems image files are becoming huge these days.

Terry
« Last Edit: June 18, 2012, 08:03:57 PM by Terry-M » Logged
aaturner
Newbie
*
Posts: 22


Email
« Reply #4 on: June 19, 2012, 08:51:16 PM »
To expand somewhat on my previous comments I have now carried out a more detailed evaluation of the time it takes to build raw cache thumbnails. My test results are as follows:
93 images with an average size of 45.5 Mb per image.
Time to load and build thumbs for "Lightning Cache": 9 mins 41 secs which averages 6.25 secs per image.
When building thumbs for full sized TIFFs the process takes significantly longer (unmeasured), but then the file size averages 117 Mb.
Nothimg else was running during the test. Does this sound about right?
Thanks for your earlier comments and ongoing support.
Logged
aaturner
Newbie
*
Posts: 22


Email
« Reply #5 on: June 19, 2012, 09:00:33 PM »
One more thing. The image editor part of the software does not like my NEF files; if I double click on a thumb I either get an "image read error" or a blank screen. Dragging and dropping a thumb onto the print window produces the required image and the photo prints ok. Does this mean that DCRaw has not yet been updated for the Nikon D800?
Logged
Terry-M
The Honourable Metric Mann
Forum Superhero
*****
Posts: 3251



WWW
« Reply #6 on: June 19, 2012, 10:25:04 PM »
Quote
Time to load and build thumbs for "Lightning Cache": 9 mins 41 secs which averages 6.25 secs per image.
If you mean that is to build thumbs and create cache, then it is quite good compared to the test  I did. Note it is 2 operations, QU tells you the progress in the bar below the menus that give the screen resolution and number of images in the folder.
Quote
When building thumbs for full sized TIFFs the process takes significantly longer (unmeasured), but then the file size averages 117 Mb.
Quote
Does this sound about right?
No!
I created some tiffs (110MB each) from the  nine D 800 NEF samples I had down loaded; the medium & best quality thumbs build very quickly, too quick to time so less than 0.5 seconds each. Thumb creation is always significantly faster than cache creation.
Quote
The image editor part of the software does not like my NEF files; if I double click on a thumb I either get an "image read error" or a blank screen
I have no problem here with editing the NEF files.
Quote
Does this mean that DCRaw has not yet been updated for the Nikon D800
I just checked the camera list on the web site and the D800 is not listed, the D700 is. All seems ok. here with the D800 files, maybe the raw format was not changed.
Are you pre-processing your NEF files in Nikon software? That could be causing these problems.
Terry
Logged
aaturner
Newbie
*
Posts: 22


Email
« Reply #7 on: June 20, 2012, 09:55:24 AM »
Something quirky going on here. I ran another test with 9 NEF images and ingested them with QIU, bypassing my usual process with IDImager. Result was images ingested reasonably quickly (at least as fast as IDI but with no metatagging and keywording) and thumbs and cache produced at a rate of about 2'' per image; i'd say that that is ok. However, the NEF files could still not be read by image editor - same outcome as earlier.

Next I converted the 9 NEF images to tiffs. Some converted successfully while others just produced a blank image and thumb. Those NEFs that successfully converted to tiffs would now open correctly in the image editor (as well as the TIFFs); those that produced blank TIFFs would still not open in the image editor.

Any ideas why this might be? Many thanks.
Logged
Fred A
Forum Superhero
*****
Posts: 5644



WWW Email
« Reply #8 on: June 20, 2012, 10:33:50 AM »
Quote
Something quirky going on here. I ran another test with 9 NEF images and ingested them with QIU, bypassing my usual process with IDImager. Result was images ingested reasonably quickly (at least as fast as IDI but with no metatagging and keywording) and thumbs and cache produced at a rate of about 2'' per image; i'd say that that is ok. However, the NEF files could still not be read by image editor - same outcome as earlier.

I downloaded one D 800 NEF file as a test.
It is a picture someone took of a black truck.

Qimage Ultimate opens and auto exposes and sets the auto fill light just as if it was from my Canon.

So what I intend to do is send you this D800 image (just accept the Download when the email arrives) and you open this in QU.
See if you get a clean well processed image. See if you get the EXIF information.
If yes, then compare this image to one that is giving you trouble. See if you can find a difference in the EXIF header especially.
If no, this image will not open and you get an Image Read Error, then make sure you have the latest version of QU. and we can go from there.

Fred
Logged
Fred A
Forum Superhero
*****
Posts: 5644



WWW Email
« Reply #9 on: June 20, 2012, 02:23:04 PM »
Quote
Any ideas why this might be? Many thanks.

The other point might be that the D 800 is not yet covered by the Raw image DCRAW,
That may cause erratic behavior as some parts of the image and header are OK and sometimes, not OK.

Just have to wait for Mr. Coffin

Fred
Logged
aaturner
Newbie
*
Posts: 22


Email
« Reply #10 on: June 22, 2012, 05:46:03 PM »
Thanks for sending me the D800 nef file of the black truck; reason I took so long in responding is that I thought that the file might be spam and sent it to the trash can. I retrieved it and opened it without difficulty; there seems to be no difference that I can tell between my files and this one. All my files open normally in PS CS6 ACR 7.1 as well as my working copy of the now extinct LightZone, so I am perplexed as to why I'm having difficulty with some files in QU (I am using the latest version). Perhaps I should just wait for DCRaw to be updated, as printing with QU isn't an issue, just using the image editor.

Andrew
Logged
Fred A
Forum Superhero
*****
Posts: 5644



WWW Email
« Reply #11 on: June 22, 2012, 05:49:43 PM »
You have the right answer!
We have to wait for the new DCRAW which will be specifically tailored to the D 800.

Thanks,
Fred
Logged
admin
Administrator
Forum Superhero
*****
Posts: 4220



Email
« Reply #12 on: June 25, 2012, 04:27:08 PM »
Update: dcraw 9.15 has finally been released.  Work is underway right now to bring the new cameras to QU.  ETA is 2 weeks but dcraw 9.15 is a major change, so don't set that date in stone.

Mike
Logged
DdeGannes
Full Member
***
Posts: 175


Retired Banker; Golf; Photography; Travel.
« Reply #13 on: June 25, 2012, 11:40:41 PM »
Quote from: Mike Chaney on June 25, 2012, 04:27:08 PM
Update: dcraw 9.15 has finally been released.  Work is underway right now to bring the new cameras to QU.  ETA is 2 weeks but dcraw 9.15 is a major change, so don't set that date in stone.

Mike

Thanks for the heads up Mike.
Logged
COMP EQP: iMac 27" mid 2015 5K Retina macOS 11.2.3; 24GB Ram; Scan Elite 5400 film scr.
CAMERA EQP: Oly OMD EM-1, Digital Zuiko & OM lenses.
Imaging Apps: PS CC 20; LR Classic CC 9.3; Qimage U & One; VueScan.
Pages: [1]
  Print  
« previous next »
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.21 | SMF © 2015, Simple Machines Valid XHTML 1.0! Valid CSS!
Security updates 2022 by ddisoftware, Inc.