File Synchronization

[almerger]

I have an EFS that reads a file that is constantly being updated by another program (TradeMaven). Sometimes the file appears to be incomplete or not correct, then it is OK again. I'm wondering if there could be a synchronization problem, where the EFS is trying to read the file while TradeMaven is writing it. Could this be the problem, and if so, do you have any suggestions for how to handle it?

[Guest]

Re: File Synchronization

Hi almerger,

Originally posted by almerger
I have an EFS that reads a file that is constantly being updated by another program (TradeMaven). Sometimes the file appears to be incomplete or not correct, then it is OK again. I'm wondering if there could be a synchronization problem, where the EFS is trying to read the file while TradeMaven is writing it. Could this be the problem, and if so, do you have any suggestions for how to handle it?

I have an EFS that reads a file that is constantly being updated by another program (TradeMaven)... - does this mean that TradeMaven is keeping the file open?

...Sometimes the file appears to be incomplete or not correct, then it is OK again... - I have seen this where the host application is not flushing the buffer when writing to a file. This would be an issue on the TradeMaven end.

...I'm wondering if there could be a synchronization problem, where the EFS is trying to read the file while TradeMaven is writing it. Could this be the problem... - I am a bit surprised you are not getting a sharing error (ditto for my first question). It has been several years since I have run into a similar situation or otherwise had any sharing issues with files. If a sharing error is not an issue, it may very well be what you described.


...and if so, do you have any suggestions for how to handle it? - I would suggest that the file be read twice, sequentially, then compare the number of lines and the content of the last line. If they are equal, you have the complete update. You will probably have to experiment with this and the timing between file reads a bit.

I hope this helps. Please provide some feedback when you figure this out.

[almerger]

Thanks, Steve. In answer to your questions:

1. I doubt TM is keeping the file open, would guess that it opens, rewrites, and closes it, but I don't know. TM rewrites the file several times a second during busy market, and the EFS reads it on every tick.

2. Good idea about buffer flush. Since I'm only seeing the problem every 15 minutes or so on average, I'm guessing that "forgot to flush" isn't the problem.

3. Since TM is constantly rewriting the file with different data, I don't think the file length and last line comparison will tell me very much as to its completeness. But I'll reconsider this if I can't get anything else to work.

On Tuesday, I'll change the EFS to open the file as read-write ("rt+") rather than read-only ("rt"). Maybe this'll invoke a sharing violation in case TM also has it open for write, although this isn't documented in eSignal's File object documentation.

[Guest]

almerger,

You are most welcome, hopefully something there will afford a solution to your issue.

Regarding documentation, eSignal is using open source JavaScript 1.5 as a basis for it's functionality. As such, the efs documentation is only a small subset of the documentation out there available for JavaScript, understandably so, at least in my opinion. Having said that, not all JavaScript commands work in the eSignal environment either, especially those regarding window manipulation and web page functionality.

Regardless, the documentation associated with efs has several references and links to Core JavaScript 1.5 documentation, and any command included in the efs documentation, if shared with JavaScript, can be supplemented by the other JavaScript references out there. If you check out the EFS Knowledgebase or the tutorial folder in my fileshare 'Functions' (links to these and others are at the bottom of this post under Handy Links: ) you will see links to documentation for Core JavaScript 1.5.

Hope this helps out as well.