Questions about the new log format

ofri · Post by **ofri** » Wed Jan 18, 2006 7:44 pm

bgannin wrote:I don't think you can just read the last X% of an XML file, you have to read it all and seek to the end.

This is exactly why lmx was made :-P

Ringo · Post by **Ringo** » Wed Jan 18, 2006 7:56 pm

bgannin wrote:This addresses display more than the file writing and management.

It addresses both display and file management. If each log file is supposed to represent a conversation, then they are two parts of the same mechanism.

bgannin wrote:In addition, you run the chance of still creating large files (i.e., thousands upon thousands of objects internal) that would need to be read in again.

The options for the timeout would have to be limited, or at least Adium would display a warning if they were set outside the recommended timeframe.

bgannin wrote:I don't think you can just read the last X% of an XML file, you have to read it all and seek to the end. Thus every conversation opened would require a log file to be read into memory, seeked to the end, checked, and then parsed for display and passed over for continued logging or generate a new file.

That is correct. Read the wiki article again. [edit: whoops, I can't find any mention of this in there!] Most of this process (reading and parsing the log file, and displaying it in the window) is already part of the plan. Instead of displaying the last n lines of the conversation, though, it would display the entire thing. I believe that Adium currently only displays a limited number of lines because the the logs aren't parse-able. Saving the entire conversation would have be wasteful and unwieldy.

Keep in mind none of this would only happen unless you've chatted with that person within a short amount of time. Unless these people are online all hours of the day talking to each other, there is no risk of the logs becoming much larger than they are now.

I was also thinking that when a conversation is about to be resumed, perhaps Adium could ask if they'd rather start a new conversation. If someone can think of a good way to do that (I haven't) then it might be useful.

Ringo · Post by **Ringo** » Wed Jan 18, 2006 8:17 pm

evands wrote:Are you saying that "Resume conversation" would output the entire previous log into the new message window?

Yup, if that's the option you choose. The other options would act like they do now.

Ringo · Post by **Ringo** » Wed Jan 18, 2006 8:30 pm

On a completely unrelated note, it appears I got post number 50000!

Post by **bgannin** » Wed Jan 18, 2006 8:51 pm

I still find this to be a soft divider, much as day is. This system will instead make assumptions (which are always devilish in programming) based on an arbitrary setting. Conversations aren't necessarily breakable by fine-grained time distinctions any more than macro distinctions such as day.

To be clear: breaking on day is no different than this except it fixes the break value at 24 hours. The described mechanism simply exposes this for user-controlability. A conversation may end when the window closes and a new topic/conversation started that would logically (to the user) be a separate log file but falls within the time range specified. Thus to be able to have finer-grain control it would require toggling the setting at times (this is for those who are that obsessive about logging... which is like 10% [stats made up, minority is the implication], versus the enforced default which makes sense to the broadest base.)

And yes, I'm aware of the article, trac ticket, dev list discussions, and how it works now. I've been in the log system a fair bit in my time working on Adium.

evands/ofri: interesting, I'd forgotten about liblmx as I think it got lost (to my mind at least) during the filestream discussion.

Post by **The_Tick** » Wed Jan 18, 2006 9:01 pm

Colin, which is better for you guys coding this, per day or per convo?

Whichever he answers, I think that one should just be used. Adding this new pref seems silly.

Post by **evands** » Wed Jan 18, 2006 9:01 pm

Ringo wrote:]I believe that Adium currently only displays a limited number of lines because the the logs aren't parse-able.

No, the default is 5 lines because adding messages to the webkit view is slow. Whether the information is parsed from the log file or from separate storage, that remains the case.

In any case, this just isn't something that needs to be under the control of the user. Displaying logs by account->contact->date makes more sense than account->contact->date->arbitrary divider; it is rare that you would think, "oh, that was the second conversation I had with John 3 weeks ago" and want to look at that level of specificity when pulling up a log. What's displayed in the message window is just that -- display -- and needn't have any dependency upon how the information is stored on disk.

Ringo · Post by **Ringo** » Wed Jan 18, 2006 10:10 pm

bgannin wrote:I still find this to be a soft divider, much as day is. This system will instead make assumptions (which are always devilish in programming) based on an arbitrary setting.

Yes it is still somewhat arbitrary, I agree, but it's less so than the window state or the current date. Window state can potentially relate to conversations; but that depends completely on their use, and not everyone uses them the same way.

A timeout would at least solve the main drawbacks of both other designs, as well as adding a feature which most people would find useful.

My coworker was complaining the other day about how iChat creates a new log file every time a new window opens, and how he always has to clear out the files because they take up too much space. The minimum file size on an HFS+ drive is 4KB. Something like log files that can add up faster than you'd think for people who frequently close windows.

When I mentioned my idea for Adium he said something I hadn't thought of as being terribly important. He said he liked it "as long as there's some sort of demarkation of when the break occurred." In other words, as long as the event:windowclose and event:windowopen tags were displayed in the log viewer, it would be very useful. I agree.

bgannin wrote:To be clear: breaking on day is no different than this except it fixes the break value at 24 hours. The described mechanism simply exposes this for user-controlability.

I disagree. If you start talking to someone at 11 PM and the conversation lasts 90 minutes, then it would be two files. This is as arbitrary as you can get as that divider is completely artificial. Further, the break value is only 24 hours at maximum, and 1 second at minimum. I wouldn't characterize that as being fixed.

bgannin wrote:A conversation may end when the window closes and a new topic/conversation started that would logically (to the user) be a separate log file but falls within the time range specified. Thus to be able to have finer-grain control it would require toggling the setting at times (this is for those who are that obsessive about logging... which is like 10% [stats made up, minority is the implication], versus the enforced default which makes sense to the broadest base.)

I agree, but the window-based approach only works if the user is actively aware of it. Further, the window-based approach doesn't take into account that the person may quit Adium temporarily such as for software update restarts, when the computer is commandeered by another person, etc.

A button or menu selection for "end conversation" would solve that very well. Activating it would close the log file, open a new one and clear the window history. I see no reason it would have to be based completely on the timeframe, and Adium doesn't need to be able to understand context.

bgannin wrote:And yes, I'm aware of the article, trac ticket, dev list discussions, and how it works now. I've been in the log system a fair bit in my time working on Adium.

Sorry about that. I didn't mean that in a condescending way. You were making it sound as if this would cause a massive amount of extra overhead when clearly the majority of that work is being done anyway.

Ringo · Post by **Ringo** » Wed Jan 18, 2006 10:22 pm

evands wrote:No, the default is 5 lines because adding messages to the webkit view is slow. Whether the information is parsed from the log file or from separate storage, that remains the case.

Shoot. How slow?

evands wrote:In any case, this just isn't something that needs to be under the control of the user. Displaying logs by account->contact->date makes more sense than account->contact->date->arbitrary divider; it is rare that you would think, "oh, that was the second conversation I had with John 3 weeks ago" and want to look at that level of specificity when pulling up a log. What's displayed in the message window is just that -- display -- and needn't have any dependency upon how the information is stored on disk.

Actually, it would be "account->contact->arbitrary divider".

There's no reason that the log viewer couldn't work as it does now. The user clicks on a specific date and it displays all logs that were started on that date. It would display them as though they were together, but mark where the boundaries are. More importantly, it doesn't have to cut them off at midnight. Instead it simply highlights where the date changed just as Adium does now in the message view.

Post by **Catfish_Man** » Wed Jan 18, 2006 10:29 pm

Ringo wrote:
evands wrote:No, the default is 5 lines because adding messages to the webkit view is slow. Whether the information is parsed from the log file or from separate storage, that remains the case.
Shoot. How slow?

Depends on the webkit version* and the computer you're running it on. Turn it up to 100 in prefs and find out.

*post-1.0 project I'd like to try: make the WKMV faster by using the new ObjC DOM API

Ringo · Post by **Ringo** » Wed Jan 18, 2006 10:31 pm

evands wrote:No, the default is 5 lines because adding messages to the webkit view is slow.

Wait wait wait. Shenanigans! Why would you have to add them individually? Just recreate the HTML as it was during the last session and pipe it into WebKit on creation of the object instance.

Post by **Catfish_Man** » Wed Jan 18, 2006 10:33 pm

Ringo wrote:
evands wrote:No, the default is 5 lines because adding messages to the webkit view is slow.
Wait wait wait. Shenanigans! Why would you have to add them individually? Just recreate the HTML as it was during the last session and pipe it into WebKit on creation of the object instance.

That would mean we'd have to save a bunch of html on window-close, which wouldn't solve our "message history lost on crash" bug.

Ringo · Post by **Ringo** » Wed Jan 18, 2006 10:48 pm

Catfish_Man wrote:That would mean we'd have to save a bunch of html on window-close, which wouldn't solve our "message history lost on crash" bug.

Ew. God no. Recreate it from the log file.

This reminds me of when the WebKit message view was still being created and I was screaming "use XML and CSS." But I'm not gonna say I told you so. Even though I did. :P

Post by **Catfish_Man** » Wed Jan 18, 2006 11:13 pm

Ringo wrote:
Catfish_Man wrote:That would mean we'd have to save a bunch of html on window-close, which wouldn't solve our "message history lost on crash" bug.
Ew. God no. Recreate it from the log file.

This reminds me of when the WebKit message view was still being created and I was screaming "use XML and CSS." But I'm not gonna say I told you so. Even though I did.

/me points at adiumxtras.com's message style section

Using skills people already have DOES pay off sometimes...

Ringo · Post by **Ringo** » Wed Jan 18, 2006 11:19 pm

Catfish_Man wrote:/me points at adiumxtras.com's message style section

Using skills people already have DOES pay off sometimes...

Yeah yeah yeah.

Just a little extra work on the developer end (which I was willing to do) and it would have been even easier than the HTML stuff. Regardless, I was joking anyway.

memark · Post by **memark** » Wed Jan 18, 2006 11:20 pm

I touched upon this earlier, without much response, so I'd like to bring it up again.

You appear to be discussing the 'division into separate log files' mainly as a question of GUI usability. (Apart from the mentioned disadvantages of having huge/many files.) But is that really the main concern? Does the 'log file implementation details' even need to be exposed to the end user in such a direct way? Shouldn't the larger question be how we want to present the combined log data to the user?

Personally I very seldom know what date (or conversation!) I discussed a certain subject. Instead I use the search field to find that out. Only after searching do I look at the date of the log. Metadata is certainly marching forward, and I hope to see extended search capabilities in the future!

Post by **evands** » Wed Jan 18, 2006 11:25 pm

memark wrote:You appear to be discussing the 'division into separate log files' mainly as a question of GUI usability. (Apart from the mentioned disadvantages of having huge/many files.) But is that really the main concern? Does the 'log file implementation details' even need to be exposed to the end user in such a direct way? Shouldn't the larger question be how we want to present the combined log data to the user?

Indeed, that was the point I was trying to make previously:

evands wrote:What's displayed in the message window is just that -- display -- and needn't have any dependency upon how the information is stored on disk.

memark wrote:I hope to see extended search capabilities in the future!

Oh, you will

Post by **The_Tick** » Wed Jan 18, 2006 11:26 pm

memark wrote:I touched upon this earlier, without much response, so I'd like to bring it up again.

You appear to be discussing the 'division into separate log files' mainly as a question of GUI usability?

Nope. The intention is that the user doesn't know about this at all, until the stuff Ringo was talking about showed up. afaik

Post by **evands** » Wed Jan 18, 2006 11:28 pm

Ringo wrote:
evands wrote:No, the default is 5 lines because adding messages to the webkit view is slow.
Wait wait wait. Shenanigans! Why would you have to add them individually? Just recreate the HTML as it was during the last session and pipe it into WebKit on creation of the object instance.

In addition to Catfish_Man's point, note that this would require message history to be tied to message styles -- change your style and your history shows up with HTML formatting from that previous style?

Ringo · Post by **Ringo** » Wed Jan 18, 2006 11:29 pm

memark wrote:You appear to be discussing the 'division into separate log files' mainly as a question of GUI usability. (Apart from the mentioned disadvantages of having huge/many files.) But is that really the main concern? Does the 'log file implementation details' even need to be exposed to the end user in such a direct way?

Somewhat. I think it would enhance Spotlight results.

I'll make a better reply later, though. Gotta run.