Mike Chaney's Tech Corner

Mike's Software => Qimage Ultimate => Topic started by: UltraChrome on September 04, 2010, 03:06:43 PM



Title: Thumb sort order
Post by: UltraChrome on September 04, 2010, 03:06:43 PM
Does QU use the time stamp as well as the date stamp when sorting? I ask because in QU when I  sort to  ~ date - ~ it does not seem to take into account the time stamp, only the date. This results in my cards downloading in a different order that the shoot. Camera is a Nikon D700. Thanks for any info you can provide. I don't remember this being a problem with Qimage Studio version.

Example:
Here's how they appear in the preview window (starting at the top left):
700_0375.nef
700_0376.nef
700_0373.nef
700_0374.nef
700_0377.nef
700_0380.nef
700_0381.nef
700_0378.nef
700_0379.nef

Per the "date created", the order should be 0381 thru 0373 with 0381 being in the top left of the preview window. It appears that QU is using the date I transferred the files from the card to my hard drive instead of the actual date created.


Title: Re: Thumb sort order
Post by: Fred A on September 04, 2010, 03:39:24 PM
I would have to mention that EDIT PREFERENCES offers you a variety of sort options.
I normally do Names +, but changed to Date +
My folder with 113 Raw images resorted by Date stamp + time (including minutes and seconds) as some were shot rapidly with 3 or 4 shots in less than a minute.
Coincidentally, the numerical filenames happen to coincide with the time stamp.
It is sorting by time stamp though, as some of the images were copied and have unique filnames.

Fred


Title: Re: Thumb sort order
Post by: admin on September 04, 2010, 04:26:54 PM
It's by both date and time so there are two possibilities:

(1) Qimage cannot read the EXIF time stamp in the files so it is using file date/time and the file date/time is not consistent with when the files were actually created on the card.

(2) The EXIF date/time in each image is not correct.

Seems that the first one is more likely.  When cameras write to flash cards, they are supposed to write the files with a file modification date equal to the EXIF date/time stamped inside each image.  Take a look at your files and look at the file date/time and see what order that shows.  Maybe it is following that, which should normally be OK unless the camera (or something else) fouled up the file modified date/time.

Regards,
Mike


Title: Re: Thumb sort order
Post by: MelW on September 05, 2010, 12:55:27 AM
By the way - for what it's worth - numerical file names don't always correspond to time sequence 100% of the time.  On my D200, if I take pictures a repid sequence (3 or 4 frames a second), there will be scattered images out of time sequence.  Has to do with the way the images are buffered before being stored on the flash memory. Other than that Fred has it - as he usually seems to.

Mel


Title: Re: Thumb sort order
Post by: UltraChrome on September 05, 2010, 02:49:28 PM
Thanks for the replies. I've modified this response several times. Suffice it to say, I can't get them to sort in the correct order using the date +/- field in Edit Preferences section. Using a date/time field showing only date:hour:minute:second will not be enough to keep the files in order. If I'm shooting at 9fps that data will be written EXACTLY THE SAME 9 times to the card. There has to be another (or an additional) field to retain correct shooting order.

As an aside, I've started a ticket with Nikon to discuss MelW's comment about file numbers not being written in the order they were shot. As I shoot motorsports I really can't have shots coming out in some random order. Maybe it was a D200 thing...


Title: Re: Thumb sort order
Post by: Fred A on September 06, 2010, 08:59:55 AM
Quote
There has to be another (or an additional) field to retain correct shooting order.

Have you tried sorting by NAMES + or - ?
Unless you are shooting Video, each shot should have a number of its own, No?

Fred


Title: Re: Thumb sort order
Post by: admin on September 06, 2010, 01:12:30 PM
File date/time is actually stored down to the millisecond so that shouldn't be a problem.  Most programs only show the second but the file modified date/time is actually accurate to the millisecond.  I doubt your Nikon is fast enough to shoot two on the same millisecond.   ;D  So it's down to: does the camera write the proper date/time (down to the millisecond)?  Maybe the camera is only writing to the second?  If so... I'd say that's the camera's fault, and possibly they can enhance in firmware.

Edit: While the time is returned to the millisecond, it appears the OS doesn't actually resolve it that fast, so we're down to the EXIF data which only resolves to the second.  Bottom line, I don't think it is possible to sort by date/time when shooting at 9 fps because the data is not there to support it.

Mike


Title: Re: Thumb sort order
Post by: UltraChrome on September 06, 2010, 04:07:06 PM
Bottom line, I don't think it is possible to sort by date/time when shooting at 9 fps because the data is not there to support it.

Mike

Correct if you are only using the info out to the second. I asked Nikon "When shooting at high speed on a Nikon DSLR (D200-300-700), is it possible that the order of the shots taken will be incorrectly named? For instance, if I shoot 10 shots in 2 seconds, will they ALWAYS be saved in the correct order, i.e., 1 thru 10? Or is it possible the order will change so that they are not named in the shooting order?"

They replied:

"In regards to your inquiry, it isn't possible that the shots get named incorrectly"

so the story about saving files by buffer order is urban legend (or a malfunctioning camera). As I said, Nikon software does not have any problems with a date sort so they must be reading some other (or additional) field(s) for the date sort function. Nikon also gives you the option of sorting by either date created or date modified, ascending or descending. So you can always put your shots back into correct shooting order by date regardless of any name changes or edits, which is what I was hoping would be possible in QU.


Title: Re: Thumb sort order
Post by: UltraChrome on September 06, 2010, 04:08:20 PM
Have you tried sorting by NAMES + or - ?
Unless you are shooting Video, each shot should have a number of its own, No?

Fred

That only works until you change a filename...


Title: Re: Thumb sort order
Post by: admin on September 06, 2010, 08:35:43 PM
They're probably just sorting by file name by virtue of the fact that they are creating the file names in order in the first place.  If the camera puts the proper order on the file names, why not just leave the original file name as part of the (end result) file name?  That's the only way you're going to be able to sort them in as-shot order anyway.  Both the EXIF date/time and the file date/time won't resolve to 9 fps.

Mike


Title: Re: Thumb sort order
Post by: MelW on September 06, 2010, 08:55:28 PM
Urban legend or not - I can tell you it has happened to me - not often - and I will have to dig a little for the examples because I don't often shoot that rapid fire - but I do have instances where the image name and the date/time sequence are different.  And how do you suppose I even noticed this?  I noticed because of Qimage of course.  When I sorted by file name, there were one or two pictures in different order than sorting by date time (one or two out of say 200).  Lates this week I will dig up the examples to make sure that I am not the one out of order.


Title: Re: Thumb sort order
Post by: UltraChrome on September 07, 2010, 04:45:06 PM
Mel - don't look them up for my sake! You might want to send them to Nikon though  :) They are the only ones that have the know-how to see the correct fields in the file as it is well below basic EXIF. The "creation date" is the sort field they use in their software and you can't change that with any basic editor I have found. It would be pointless to post anything here since the visible filename is changeable at will.

As for the issue, I'll just keep sorting, and editing, in Nikon NX2 and transfer only those I wish to print to QU. Unfortunately, that kinda defeats MY purpose of going from Studio to Ultimate.


Title: Re: Thumb sort order
Post by: Terry-M on September 07, 2010, 05:04:50 PM
Quote
Unfortunately, that kinda defeats MY purpose of going from Studio to Ultimate.
If this is just because of sort order and your current desire to re-name some images, how about re-thinking the re-naming policy?
This is what I do fwiw:
I never re-name a raw file except as it's copied from the flash card and always keep the camera file number in the file name. The first part of the name is the file date (yy-mm-dd when it was taken) and I sort by name. Any files converted from the raw are nearly always in a separate folder and would get renamed there as required.
Terry


Title: Re: Thumb sort order
Post by: UltraChrome on September 07, 2010, 05:38:29 PM

I never re-name a raw file except as it's copied from the flash card and always keep the camera file number in the file name. The first part of the name is the file date (yy-mm-dd when it was taken) and I sort by name. Any files converted from the raw are nearly always in a separate folder and would get renamed there as required.
Terry


If you only have the yy-mm-dd in your name sort, how do you keep it in correct shooting order? That's not enough info unless you only shoot ONE photo per day! I'm dealing, theoretically,  with shoots that typically have over 1000 shots per day, occurring in 50-75 frame bursts (such as a NASCAR road race in a given corner - although I shoot motorcycles, mostly  :))...


Title: Re: Thumb sort order
Post by: Terry-M on September 07, 2010, 06:02:28 PM
Quote
If you only have the yy-mm-dd in your name sort, how do you keep it in correct shooting order?
I said: "and always keep the camera file number in the file name"  ::)
Does your camera not keep those in sequence  ???
I was adding to what Fred said & your reply:
Quote
Have you tried sorting by NAMES + or - ?
Unless you are shooting Video, each shot should have a number of its own, No?
Fred

That only works until you change a filename...

It just seemed a poor reason not to take advantage of QU; there's always other way of approaching a problem.
Oh and btw,
Quote
That's not enough info unless you only shoot ONE photo per day!
That is definitely not me.

Terry


Title: Re: Thumb sort order
Post by: UltraChrome on September 07, 2010, 09:53:34 PM
Thanks for the clarifications and comments. I really was just curious as to why QU wouldn't sort by date correctly - I guess the answer is "that's the way it is and the way it will stay". Not really a big deal, but it does nullify the date sort function.


Title: Re: Thumb sort order
Post by: admin on September 07, 2010, 11:12:58 PM
Thanks for the clarifications and comments. I really was just curious as to why QU wouldn't sort by date correctly - I guess the answer is "that's the way it is and the way it will stay". Not really a big deal, but it does nullify the date sort function.

Just to clarify, QU does sort by date (and time) properly!  It's the simple fact that you do not have the data to support time sorting when shooting at 9 fps.  The operating system(s) don't support it and neither does EXIF.  That is not a Qimage issue.

P.S.  There are easy ways to resolve this BTW.  Use QU's auto-rename function to rename the files by date and time and at the end just include the numeric part of the original file name.  Then if you have 5 pictures that were taken on the same second, you'll get file names like
2010-09-07 13.46.52 [0374]
2010-09-07 13.46.52 [0375]
2010-09-07 13.46.52 [0376]
2010-09-07 13.46.52 [0377]
2010-09-07 13.46.52 [0378]

And they'll always be in sequence as long as your camera numbers them sequentially.  Then just sort by filename and you get the proper sort.

Mike


Title: Re: Thumb sort order
Post by: UltraChrome on September 08, 2010, 03:24:47 PM
Just because I feel like arguing with you:

The sort function of QU does NOT work for date. To have to rename a file in order for a function to work is, to me, the same as not working. As I have said before, Nikon Capture and View BOTH sort correctly on date, every time, every frame, no matter the frames per second, WITHOUT modification, over 50,000 frames sorted from FOUR different Nikon DSLR cameras, so it IS POSSIBLE and the data IS THERE.

End of rant....


Title: Re: Thumb sort order
Post by: admin on September 08, 2010, 03:34:08 PM
Just because I feel like arguing with you:

The sort function of QU does NOT work for date. To have to rename a file in order for a function to work is, to me, the same as not working. As I have said before, Nikon Capture and View BOTH sort correctly on date, every time, every frame, no matter the frames per second, WITHOUT modification, over 50,000 frames sorted from FOUR different Nikon DSLR cameras, so it IS POSSIBLE and the data IS THERE.

End of rant....

Just because I know for a fact you are wrong:

QU absolutely DOES sort properly by date/time.  Date/time is defined as the file modified date/time or the EXIF date/time neither of which contain data that can resolve 9 fps.  If the Nikon software is sorting in sequence, then they are either sorting by date PLUS filename (which is a different sort entirely) or they have a proprietary timecode in the file that only Nikon can read (which isn't very useful).

Mike


Title: Re: Thumb sort order
Post by: UltraChrome on September 08, 2010, 03:38:31 PM
I'm guessing Nikon is using the shutter count field for the correct order. Changing the name of a file NEVER interferes with the correct date sort I get on Nikon software. Just curious - is this a problem only with Nikon or do other file formats not sort correctly if shots are fast?


Title: Re: Thumb sort order
Post by: admin on September 08, 2010, 04:28:10 PM
I'm guessing Nikon is using the shutter count field for the correct order. Changing the name of a file NEVER interferes with the correct date sort I get on Nikon software. Just curious - is this a problem only with Nikon or do other file formats not sort correctly if shots are fast?

It's a problem with any camera that shoots that fast since both the file date/time and the EXIF date/time stamp will be the same if you shoot faster than once a second.  Nikon must be getting their data from a proprietary "maker notes" tag in the file.  That's not something I plan to support but I *can* make it so that if the date/time stamp is the same for a bunch of files, it keeps the file names in order.  In other words, I can make the file name the secondary sort.  I think that would do it for most people.

Mike


Title: Re: Thumb sort order
Post by: UltraChrome on September 09, 2010, 03:01:02 PM
It would be good to at least keep the "seconds" in order. Thanks for listening. Here's my latest "date" sort:

Without going into the entire sequence of shots, here’s how it sorted when I selected date/time. The sort was – (minus) and should list LAST SHOT FIRST:

File Name      File Date Time (hh:mm:ss)
1202         09:24:22
1201         09:24:21
1200         09:24:21
1205         09:23:23
1204         09:24:23
1203         09:24:22
1197         09:24:20
1196         09:24:19
and so on.

Order should be (from first to last):
1205-1204-1203-1202-1201-1200-1199-1198-1197-1196

The numbers missing from the above example are further down in the preview window. Obviously the sort is way off.


Title: Re: Thumb sort order
Post by: admin on September 10, 2010, 01:51:08 PM
The operating system only supports file date/time down to ~2 second intervals, so Qimage is bound by that.  That said, I think you must be looking at the wrong column.  Where are you getting your file times?  Explorer no longer shows seconds (because MS knows the OS cannot resolve to the second), so where are you getting your file times?  Qimage sorts by file modified date.  Maybe you are looking at file creation date or some other date?  It definitely will not sort 9:24:22, then go to 9:24:23, and back to 9:24:19.

In fact, something is very "funky" with your column that you list as "file date time" because if you list the actual file date/time, you won't see a combination of even and odd seconds.  So I know you got your data from some place other than the actual file date/time.  Did you get it from the Nikon data?

Mike


Title: Re: Thumb sort order
Post by: admin on September 10, 2010, 03:23:47 PM
I just released 118 with a tweak for sorting by date so give that a try.  It does a subsort by filename which should take care of 99% of issues with shooting faster than your system can record the file date/time.

Mike


Title: Re: Thumb sort order
Post by: UltraChrome on September 11, 2010, 07:23:02 PM
I've tried the new version. It looked promising when I tried sorting 12 files in "date +" order. Then I sorted in "date -" order and the order did not change at all. Then I tried a larger directory area (containing jpg, tiff and raw files, and it seems the date sort is TOTALLY unusable. It seems to have no order at all, and, worse yet, it won't even retain the same incorrect order if you select between + and - (i.e. it does not "mirror" the order) . It IS consistent in its inconsistency, however. I think some problems (although not all) may have to do with the use of special characters. The Nikon naming system uses an underscore "_" as the first or fourth digit to determine the color space used. For example, if I shoot sRGB, the file will be named _xxxXXXX (with the xxx being selectable by me using any character and XXXX being the 4-digit frame number assigned by the camera). If I shoot AdobeRGB, the file will be name xxx_XXXX.

I did go back and reload Qimage Studio and I notice it has the same problems I originally posted here (but that's better than whatever you did to Ultimate). I never noticed it before but was not using QI Studio the same way I would like to use Ultimate.

As for what field I use for my data given in my posts, I have been using Nikon's "date shot" field.  Their software  displays "created"; "modified" and "shot" dates/times. The "date shot" shows milliseconds, both in Nikon ViewNX and Nikon Capture NX2. FWIW, I haven't really dealt with any programming since in the 60's when we used punchcards in Fortran or Cobol...

EDIT For my info, what are the hex locations of the fields you are using for the date sort (it looks like you use the one starting on line 400 which is the second date I see)? It may help me shake this down. All I want to do is get a correct  date by milli-second sort! Am I the only one that shoots more than 1fps?

EDIT (file removed) See the attached file for HEX locations...not really sure if that helps as that location seems to change...


Title: Re: Thumb sort order
Post by: Terry-M on September 11, 2010, 09:59:32 PM
Quote
Am I the only one that shoots more than 1fps?
Last week I went to a Show Jumping event and used continuous shooting for much of the time. I'm not up to your 9fps but was at 3fps with my modest Canon. I was shooting in raw so kept the bursts down to no more than 6 frames to allow for writing. Horses are not as quick as cars either ::)
Prior to Mike modifying QU to v118, I had no problem because I use FlashPIpe to re-name in the form yy-mm-dd #camera file number and sort by name +.
With v118 sorting by date + or - is fine too. The file data shows I was getting up to 3 images with the same time to the second as expected and all are in sequence.
How much difference 3 to 9 fps makes or Canon compared to Nikon with respect to sorting I don't know, but it works for me here.  :)
Terry


Title: Re: Thumb sort order
Post by: UltraChrome on September 11, 2010, 10:19:43 PM
RE:
With v118 sorting by date + or - is fine too. The file data shows I was getting up to 3 images with the same time to the second as expected and all are in sequence.


I assume the sort is done AFTER you have renamed every file? What happens to your sorting if you just transfer the files directly to disk from the CF (or whatever) card in your camera without a rename? Does the "date sort" sort them correctly, + and - ? Thanks.


Title: Re: Thumb sort order
Post by: UltraChrome on September 11, 2010, 11:14:09 PM
For further testing, I shot two sets of shots today. SET ONE was numbered from 300_1224 to 300_1238; then two hours later I shot SET TWO numbered 300_1239 to 300_1253. FWIW, they were not shot in the same "minute". They were shot and named numerically in order from 1224 thru 1253 with correct time stamps according to Nikon ViewNX.

When sorting in QI Ultimate, if I sort by "name +" they appear correctly (#'s 1224-1253). If I sort by "name -" they appear correctly (#'s 1253-1224).

If I sort by date +, they appear as follows: #1239-1253 then #1224-1238. Sorted by "date -" changes NOTHING (no reversal of order, they show up EXACTLY the same way as the "date +" sort). In "date +" they should appear #1224-1253 (same as if sorted by "name +"); in "date -" they should appear "#1253-1224" (same as "name -"). It looks as if it tried to sort them in a "date -" order but messed up the date order within the "set".


Title: Re: Thumb sort order
Post by: admin on September 12, 2010, 03:44:00 PM
v2010.119 contains the final tweak for date sorting.  It sorts properly by date and time (as it always has) and then sorts (forward or backward as appropriate) by file name for files that have the exact same date/time stamp.  v2010.118 sorted like Windows Explorer which isn't ideal because Windows Explorer always sorts in ascending order by file name if the date/time stamp are the same even if you told Explorer to sort by date(-).  v2010.119 works better WRT photos because it takes into account that higher numbered photos are shot later in time... something Explorer won't do and can't really assume.

If you want to compare Qimage to something, compare it to Windows Explorer.  It now does a better job than Explorer.  Forget about Nikon products.  I will not be supporting proprietary data where you have to open the file and search inside each file to find a proprietary time in the milliseconds.  Now you get a sort that is better than Explorer and that's as far as I'm going to go and quite honestly more than is needed.

BTW, if you shot a sequence 24-38, waited even a few seconds, and then you shot 39-53 and you got 39-53 first and 24-38 second on a date(+) sort even with 118, your date/time file stamps are WRONG!  Use Explorer to confirm this: Windows Explorer will show you that the date/time on the 39-53 set is before the date/time stamp on the 24-38 set.  I presume that can happen if you did something like modified or somehow "recreated" the 24-38 set after the 39-53 set.

Mike


Title: Re: Thumb sort order
Post by: UltraChrome on September 13, 2010, 04:37:16 PM
Final comment:
I shot in continuous order from 1224 through 1253. 1224 - 1235 were shot at 9am; 1236 - 1253 were shot at 2pm. The correct order for a sort should be either 1224 (oldest) through 1253 (newest) OR 1253 (newest) through 1224 (oldest). That's the order they came out of the camera and that's the order they were shot in (I shot photos of a clock with seconds to be sure they were in the correct order as they came out of the camera (as if my Nikon would misnumber them  ;D) and they were NOT shot within a single second). The files have not been modified by me in any way. Since it is obvious that you can't/won't fix this problem, I'll stop bugging you (pun intended). 'Nuff said...

Version 119: date/time sort: still incorrect

Test file #’s : 300_1224 (FIRST SHOT) through 300_1253 (LAST SHOT) continuous as shot in camera (30 shots)

Name sort works correctly (prefix "300_" dropped for clarity):
“Name +” shows 1224 thru 1253 continuously
“Name –“ shows 1253 thru 1224 continuously (direct reverse order)

“Date/time +” sort (which should be in the "name +" order: 1239:1240:1241:1242:1243:1244:1245:1246:1247:1248:1249:1250:1251:1252:1253 (LAST SHOT):1224 (FIRST SHOT):1225:1226:1227:1228:1229:1230:1231:1232:1233:1234:1235:1236:1237:1238

“Date/time –“ sort (which should be in the "name -" order::
1238:1237:1236:1235:1234:1233:1232:1231:1230:1229:1228:1227:1226:1225:1224 (FIRST SHOT):1253 (LAST SHOT):1254:1253:1252:1251:1250:1249:1248:1247:1246:1245:1244:1243:1242:1241:1239

“Close, but no pizza” I have attached two files for your perusal...

BTW - Explorer sorts them fine, + or -, using "date created".


Title: Re: Thumb sort order
Post by: UltraChrome on September 13, 2010, 06:20:28 PM
EPILOGUE:
Mike – you may be happy to know that I’ve now messed up my sorts in Explorer AND Nikon!! By renaming my files, NO date sort works right (which is what I was afraid would happen with new filenames). In the attached screen prints, the LAST DIGIT in the filename is the CORRECT sort order, 1 through 30 or 30 through 1. I’ve now got a ticket in to Nikon  <sigh>…

I see now that I have no choice but to acquiesce to renaming my files to include a reference to date/time/order shot in the first fields. It seems no programmer can actually accomplish this seemingly simple task – SORT BY DATE TO INCLUDE MONTH-DAY-YEAR-HOUR-MINUTE-SECOND-MILLISECOND. Why, then, capture that information if you cannot use it?

- 30 -


Title: Re: Thumb sort order
Post by: rayw on September 13, 2010, 06:24:55 PM
I've no idea about how Mike does what he does, but it looks to me as if it may be that the date is being sorted after being converted to another format i.e. 2pm instead of 14.00hrs when the file number/name is added. It may be, Ultrachrome, that it works OK if you say do a batch at 2pm and 4pm, say, or you change the format of the date in the exif info (I do not use ultimate, and not that area of studio, but it seems the exif date format can be changed there.

Sensibly, you should be able to sort, and sub-sort on any of the real exif data, by simply enter the tags, and/or on the iptc and/or gps data too ;).

Best wishes,

Ray

ps You posted just before me - so above may not be at all relevant, but may be worth you trying more.


Title: Re: Thumb sort order
Post by: admin on September 13, 2010, 08:26:03 PM
No, there's simply no issue with the date sort in 119.  I don't sort by "am", "pm" or anything like that.  File dates are stored as integers by the operating system: no way to get them out of order.  I will not be making any more changes because 119 works perfectly.  If it isn't working for you, Ultrachrome, then your file modified dates/times are wrong as I said before.  If you want to post any further info, please post a screen shot of Windows Explorer showing the 1224-1253 files and sort by the column "date modified" in Explorer.  Better yet, enter the folder name in the address bar in FireFox and look at the date/time that way because FireFox will show down to the second on the date/time stamp.  I guarantee you that if you look at the date modified column (which is how Qimage sorts), you'll find that 1224-1238 have a later date modified than 1239-1253!  That's what I've been trying to tell you since 118.

Mike


Title: Re: Thumb sort order
Post by: admin on September 13, 2010, 08:50:27 PM
P.S.  I just recreated your scenario.  I created 15 images (1224 through 1238) all of which had the exact same file modified date of 9am on Sept 11.  I then created 15 more (1239 through 1253) all of which had the exact same file modified date of 2pm on Sept 11.

Date(-) produced the sort: 1253,1252,1251,1250,1249,1248,1247,1246,1245,1244,1243,1242,1241,1240,1239,1238,1237,1236,1235,1234,1233,1232,1231,1230,1229,1228,1227,1226,1225,1224

Date(+) produced the sort: 1224,1225,1226,1227,1228,1229,1230,1231,1232,1233,1234,1235,1236,1237,1238,1239,1240,1241,1242,1243,1244,1245,1246,1247,1248,1249,1250,1251,1252,1253

Keep in mind that all 15 of the first set had the same 9am stamp (as if all 15 were shot at the exact same instant in time) and all 15 of the second set had the 2pm time stamp yet it still sorted properly.

It works correctly.  It sounds to me like the issue you are having is that you are confusing file creation date with file modified date.  I can't say whether or not your Nikon software has a problem because I don't know all the steps you took (there may have been other steps involved) but at first glance, it looks to me like the Nikon software is using the correct file creation date but isn't properly setting the file modified date.  Again, use FireFox to compare because it does it right (like Qimage).  It always shows you file modified date (whereas explorer sometimes just shows "Date" which is the file creation date) and FireFox also shows you seconds where Explorer does not.

Mike


Title: Re: Thumb sort order
Post by: UltraChrome on September 13, 2010, 09:30:41 PM
So you are saying I cannot sort by the date/time that the photo was taken?? I'm just looking to sort in an order that the photos were CREATED, not modified, loaded, edited or whatever. Here are two photos of the problem, taken today from 13:44:36:531 until 13:44:44:765 (approximately 8 seconds worth of shots). I have renamed the files with a random name AND THE LAST NUMBER (prior to “dot” NEF) IN THE FILE is THE CREATED ORDER (#'s 1-15). The first shows the order that is created in QIU when sorted DATE MINUS (so it sorted them 8-15-12-4, etc). The second shows those photos in order. Sorry for the loss of detail - it is the best resize I could get with the file size limitation. I can email you a beautiful jpg if you wish.

Notice the photos are a DIGITAL CLOCK WITH MILLISECONDS. If you look at the second photo, you will see that the files ARE NOT IN THE CORRECT DATE/TIME order (the order they were TAKEN in).

In reality, the sort is of no use, period. And these photos were shot within 8 seconds of each other. Your sort becomes more random as file names are changed and the size of the directory grows. It’s not in any sensible order that would lend itself to being able to find a particular photograph for a particular time frame. Nikon software will keep the CREATION DATE sort within the SECOND (even with name changes) but fails the “millisecond” test. Oh well…


Title: Re: Thumb sort order
Post by: admin on September 13, 2010, 09:59:56 PM
No, as explained previously, the only software able to show you the exact time taken is the Nikon software.  Qimage has the same data available to it as any other file manager including Windows Explorer, FireFox, and other third party tools.  The dates/times available are file "date created" and file "date modified".  Those are good to a resolution of 2 seconds.

The proper date/time sort when dealing with files is "date modified".  Qimage (like other tools that offer a date/time sort) uses "date modified" for good reason.  If you download 30 images and you then modify the first 15 and resave them, the age of the first 15 files is now later than the second set of 15 that you never modified.  That's the definition of file "age": it's the last date that the file was changed.

So let me ask you this.  Why would you download 30 files and then take the first 15 and modify them and resave them, thereby updating their age to a later time (later in fact than the second set of 15)?  Is that what you did with those 30 files?  Because you shouldn't have a problem unless you are modifying your original files or the Nikon software doesn't know how to properly set "date modified".

Bottom line: you should not be modifying NEF files so the date created and the date modified should be identical.  I'd love to know what you are doing that is screwing up the date modified.  Whatever it is, just stop doing that and it'll work fine.  Sorting by file creation date is useless.  People want to see files sorted by the last time they were updated, and that's how Qimage sorts and it sorts (always has) properly.

Mike



Title: Re: Thumb sort order
Post by: UltraChrome on September 13, 2010, 11:07:01 PM
re: "Bottom line: you should not be modifying NEF files so the date created and the date modified should be identical.  I'd love to know what you are doing that is screwing up the date modified.  Whatever it is, just stop doing that and it'll work fine."

The real bottom line is that QIMAGE is changing the data. Here's a screen shot of the files used for my last example Note the date/times. These files were DIRECTLY downloaded from a CF card using the card copy function in Ultimate. I DID NOT TOUCH OR CHANGE ONE THING, PERIOD!

So what's up?
BTW - apparently, Nikon uses what Explorer calls the "date modified" for the "date shot" in their software. Are we confused yet? That field appears to be unchangeable by those of us who don't know how to program  :D  If things are like they were in the 60's when I WAS programming, the next thing the software engineer says is "I'm blaming the hardware"!


Title: Re: Thumb sort order
Post by: rayw on September 13, 2010, 11:08:01 PM
Mike,

I don't know how you can know that 'People want to see files sorted by the last time they were updated'. Some people do, some don't, and obviously Ultrachrome wants the sort by time shot. I can see that being more useful if taking fast action shots, or working on images taken by different cameras by other people at a similar time - in which case sorting purely on the image name order would be useless (and unless the cameras were synchronised wrt time it would still be tricky, I guess, but a time offset could be applied  if you really wanted to solve the problem). I use Opanda iexf (free from http://www.opanda.com/en/iexif/) if necessary to view most exif data.

Also, for some cameras with the raw file not being handled by dcraw, the files  have to be processed by other software, so any relationship to 'date/time of original' to 'date/time digitised' will be lost.

Personally, I would have thought it not too difficult to give the option of using any of the time fields, or indeed any other tags for sorting. For example, gps info sort could be very useful, if available.

Best wishes,

Ray  

PS, Utrachrome  has done it again,  :D, pre-empted my post. Hi, Utrachrome, download opanda, compare the times shown for your images.


Title: Re: Thumb sort order
Post by: UltraChrome on September 13, 2010, 11:23:08 PM
Ray - re: "Hi, Utrachrome, download opanda, compare the times shown for your images."

I need to buy Opanda Pro for RAW files. Really don't need another EXIF viewer. I normally use Irfanview which works fine with all file formats I use. Thanks for caring  ;)

Here's the EXIF data from Irfanview for the first shot in the series:
Filename - 300_1254.NEF
ImageWidth - 160
ImageLength - 120
BitsPerSample - 8 8 8
Compression - 1 (None)
PhotometricInterpretation - 2
Make - NIKON CORPORATION
Model - NIKON D300
StripOffset - 120320
Orientation - Top left
SamplesPerPixel - 3
RowsPerStrip - 120
StripByteCount - 57600
XResolution - 300
YResolution - 300
PlanarConfiguration - 1
ResolutionUnit - Inch
Software - Ver.1.10
DateTime - 2010:09:13 13:44:59
Artist -
ReferenceBlackWhite - 0
Copyright -
ExifOffset - 600
DateTimeOriginal - 2010:09:13 13:44:59
ExposureTime - 1/50 seconds
FNumber - 5.60
ExposureProgram - Normal program
ISOSpeedRatings - 200
DateTimeOriginal - 2010:09:13 13:44:59
DateTimeDigitized - 2010:09:13 13:44:59

ExposureBiasValue - -1.00
MaxApertureValue - F 5.66
MeteringMode - Multi-segment
LightSource - Auto
Flash - Not fired
FocalLength - 105.00 mm
UserComment -
SubsecTime - 93
SubsecTimeOriginal - 93
SubsecTimeDigitized - 93
SensingMethod - One-chip color area sensor
FileSource - Other
SceneType - Other
CustomRendered - Normal process
ExposureMode - Auto
White Balance - Auto
DigitalZoomRatio - 1 x
FocalLengthIn35mmFilm - 157 mm
SceneCaptureType - Standard
GainControl - None
Contrast - Normal
Saturation - High
Sharpness - Hard
SubjectDistanceRange - Unknown

GPS information: -
GPSVersionID - 2.2.0.0

Maker Note (Vendor): -
Data version - 0210 (808595760)
ISO Setting - 200
Image Quality - RAW
White Balance - AUTO
Focus Mode - AF-S
Flash Setting - NORMAL
Flash Mode -
White Balance Adjustment - 0
White Balance RB - 706
Exposure Adjustment - 68608
Thumbnail IFD offset - 8670
Flash Compensation - -67041792
ISO 2 - 200
Lens type - 234881024
Lens - 854
Flash Used - Not fired
Bracketing - 1114112
Contrast Curve - I0
Noise Reduction - OFF
Total pictures - 4373


Title: Re: Thumb sort order
Post by: admin on September 13, 2010, 11:38:39 PM
The real bottom line is that QIMAGE is changing the data. Here's a screen shot of the files used for my last example Note the date/times. These files were DIRECTLY downloaded from a CF card using the card copy function in Ultimate. I DID NOT TOUCH OR CHANGE ONE THING, PERIOD!

Wrong!  Qimage does not change the date modified.  Check your CF card and the date modified on the CF card will match the date modified after Qimage copies it.  If they don't match, then something else on your system is changing it and very likely this could be the cause of all your problems.  I wonder if some virus software or other software you are using to "touch" the files might be causing it?  Qimage has been sorting properly by date for over a decade and no complaints, so I don't think you've all of a sudden discovered a critical flaw.

I don't know how you can know that 'People want to see files sorted by the last time they were updated'. Some people do, some don't, and obviously Ultrachrome wants the sort by time shot. I can see that being more useful if taking fast action shots, or working on images taken by different cameras by other people at a similar time

Any software that offers one date sort will sort by date modified.  The date the file is created isn't particularly important for most users and if files are copied/used properly it won't make any difference anyway.  File creation date can be fouled up simply by restoring or copying data from another drive or computer.  I'm not saying that one person wouldn't benefit from having a "date created" sort but it is not the norm.  Most people want to sort by the time the file was last modified because they want the stuff they worked on most recently to bubble to the top (or bottom).  Sorting by file creation date is simply an oddball request and I state again, completely unnecessary because "date created" and "date modified" should be identical for files created on a CF card.

Honestly, this whole thread has gone beyond ridiculous at this point.  Qimage works properly, always has, and has been modified to even be able to sort files with identical times.  On top of that I've given a myriad of different ways to completely solve the problem.  

(1) For example, when initially copying the files from the CF card, use Qimage's auto renaming capabilities to stamp the file NAME with the correct date and time followed by the original file name.  This does two things.  First, it embeds the original date/time on the file so that it CANNOT be lost by things like modifying the file or even restoring the file from a backup.  Second, it preserves the original file name if that's important to you: very easy to create files like 2010-09-11 14.00.56 (300_1238) and so on.

(2) If you simply refuse to do it the right way by making your file names have some meaning (1 above), then keep the file names.  Ultrachrome already said they sort in sequence if you simply sort by name.

I don't see the problem.

Mike


Title: Re: Thumb sort order
Post by: UltraChrome on September 14, 2010, 02:19:01 AM
re: " If they don't match, then something else on your system is changing it and very likely this could be the cause of all your problems." This sounds like the 2010 version of blaming the hardware (we only had punch cards in the 60's so we couldn't use the "virus" excuse...

Here's a screen print of the files:

Frame one shows the files as they are on the CF card (note that the "modified" and "created" dates are the same and the "date accessed" has defaulted to the same date with a 12:00 AM time (since it has not yet been accessed). This is a function of every Nikon DSLR I own.

Frame two shows the same fields after being transferred using Qimage. Note the field changes: "modified" is the same; "created" and "accessed" have changed to the time of the transfer. THIS IS INCORRECT AS THE “CREATED” DATE HAS NOT CHANGED, the files have only been transferred to a different medium. You’re re-writing the “date created” the same as Explorer does. I suppose you can argue that by placing it on the hard drive it has been “created” but that’s not how I see it. I believe a NEW DATE/TIME is “CREATED” when you edit the photo and CREATE a NEW ONE!

Frame three shows the same fields using Nikon Transfer. Note the changes. "Modified" and "created" remain the same; ONLY "accessed" changes to the time of the transfer. THIS IS CORRECT as the file has only been “accessed” for the transfer and has not been changed since it was shot with the camera. The “created” and “modified” date will change when the file is edited since it will now be a “new” file, hence, a new creation date. Their software also shows a “date shot” that NEVER changes and can be sorted within ViewNX or NX2, although only to the second (proprietary info, no doubt).

So I don’t have a virus, and yes, Qimage does have a minor flaw. It shouldn’t change the “creation” date of a file that has NOT BEEN CHANGED IN ANY WAY SINCE ITS CREATION (it’s only been “moved”). And, sadly for me, there is no way to sort the original shooting date. I’m sure the reason I never noticed this before is because I was using Qimage only to print edited files. When I started transferring and editing my photos using Ultimate I noticed the problem. Since I have no interest in changing my workflow, which has worked for me since the advent of digital, I’ll learn to deal with the flaw, whatever it requires from me (and thanks to all for the possible options). I really do like some of the functions in Ultimate and hope to learn to like it even more. Thanks for your time and patience.

PS – With all of the changes you have made, the sort is now almost non-functional. Can you put it back to the beginning?? and, PLEASE, DO NOT TAMPER WITH STUDIO’S SORT FUNCTION!!!!!!

PS - don't read Alan’s post :) He must have a virus, too….


Title: Re: Thumb sort order
Post by: rayw on September 14, 2010, 03:32:29 AM
Hi Utrachrome,

I see what you are getting at, I think. The exif dates are not altered by Qimage, or anything else if dealing with a raw image. These are the dates, afaik, that Mike uses to sort on, I hope. However if you move or copy any file, then windows will correctly assign a new creation and access date, since in the folder you move it to, it is a new file. If you copy it to the same folder, windows does not change the creation date or access date of the original file, just the new one. I would not expect it to be any different. The modified date stays the same, indicating that the contents of both files are identical (or it took less than a second to change one of them. ;-). This basic windows side of file dating pays no attention to the type of file, at the windows level, it is merely a chunk of data.

For a raw file, unless you go in and actually modify the data, then the modified date will be the same for all instances of that file. The creation date is unlikely to be the same as any date in the exif information, it is merely the time at which the file was saved. the pc clock is unlikely to be the same time as the camera clock, in any case.

Are we talking at cross purposes here ::)

Best wishes,

Ray

Best ignore the foregoing - it's too late for me to test it thoroughly.


Title: Re: Thumb sort order
Post by: admin on September 14, 2010, 03:36:57 AM
re: " If they don't match, then something else on your system is changing it and very likely this could be the cause of all your problems." This sounds like the 2010 version of blaming the hardware (we only had punch cards in the 60's so we couldn't use the "virus" excuse)...

What I was actually proposing was that some ANTI-virus software was opening the files to check them and might be erroneously resetting the date/time, not that you HAD a virus!  Now that you posted the full picture, I can see that Qimage is transferring correctly, i.e. the file modified date is carried over as it should be and the creation date is changed, just as it should be when a file is created in a new location.  That is the standard and as you observed, the way Windows does it.

Quote
You’re re-writing the “date created” the same as Explorer does. I suppose you can argue that by placing it on the hard drive it has been “created” but that’s not how I see it. I believe a NEW DATE/TIME is “CREATED” when you edit the photo and CREATE a NEW ONE!

Well I guess Bill Gates and Mike Chaney are wrong and we should just to it the "Ultrachrome way".  Please!  I've given you the proper technique for sorting by date.  You rename your files to date/time plus the numeric part of the file name.  End of story!  That method sorts properly EVERY time and you retain the ORIGINAL file date/time as part of the file name so that any program can read it and sort by it.  If you want to fool around with file creation dates, file modified dates, and hope that no one ever modifies the file, restores it from a backup, or does anything that might change those dates, just realize the consequences.  Rename your files to something meaningful and include the date/time in the file name, else you are subject to the whim of ANY program that copies, moves, or modifies files.  That's just the nature of the beast.  In the mean time, Qimage is sorting properly by the file modified date/time.

Mike


Title: Re: Thumb sort order
Post by: admin on September 14, 2010, 04:45:10 AM
I did find a problem where certain files could get sorted improperly in 118 and 119.  The fix is in 120.  It doesn't change anything WRT the discussion on what dates to use when (date modified, date created, etc.), but it does correct a problem what was throwing off the sort on 118 and 119.  120 will now sort properly by date modified (like all prior versions) plus the new file name subsort for files that have the same date/time stamp.

Mike


Title: Re: Thumb sort order
Post by: UltraChrome on September 14, 2010, 05:10:46 AM
Well, 120 looks pretty good. At least it now shows the same order ("last edit") I see in my directory on the disk, which I can live with. At least it is back on par with Studio. The basic "problem" is that you and Mr. Gates think like programmers; Nikon and I think like photographers. Gee - it was a software glitch  :D in the last two versions! Who'd a thunk it! And I had another 4 screen shots to send you from version 119...

PS - and, as a matter of clarification, I DO NOT habitually mqke ANY changes using an editor. That only started when I determined that Qimage was not able to sort the same way my Nikon software can.

Thanks for the changes and you're welcome for the debugging....


Title: Re: Thumb sort order
Post by: admin on September 14, 2010, 01:06:24 PM
I'm happy to think like a programmer and not a photographer.  If I was thinking like a photographer I might ask why you need to habitually shoot 9 fps just to get one good shot.   ;D  No, all joking aside, Qimage has to cater to the needs of more than just the photographer.  It has to work for the photojournalist, casual shooter, graphic artist, and a lot of other people that will be using it to do a lot more than sorting images shot so fast that the OS can't keep up.  It's fine that Nikon made customized software that must open each file and interrogate the contents: fast cameras will benefit from that.  But as a programmer, I must work with the data I have.  Qimage already sorted properly by date prior to 118 when you claimed it didn't and you posted those erroneous "file dates" that were not really file dates but rather the Nikon embedded shot times (reply 21 in this thread).  I think 120 is better in that it sorts more logically for photos that have the exact same date/time stamp but given that you are the only one to complain in 12 years, I doubt it'll get used by more than a handful of people.

So thanks again for the feedback and you're welcome for customizing Qimage just for you... even if there were a couple bumps along the way.

Just glad to be able to close the case on this one.

Mike