open() fails - in bizzare way

[Dave180]

Hi. Please help, I am very confused. This snippet shows problem occurring in longer script. Basically I can't seem to get sym() to work inside open() and produce a series. Oddly if I follow the failing line with a line that should be wrong (it has a ticker, but no sym() round it), the failing line starts to work. That is just bizarre. I know I'm new, but how can I get this so wrong?

Next day - a PS: On another machine I get variable results, the first time I ran this I got the "nul" debugprint. I added the "should be faulty" line (the one without the sym()) and both lines worked. I removed the faulty line and the original line now prints the numeric value. What is going on?

var bStudyInit = false;

function preMain() {
    setStudyTitle("Barmy1");
    setCursorLabelName("Barmy1");
}


function main() {

     if ( bStudyInit == false ) { // (aside - Why not use "getBarState() == BARSTATE_ALLBARS" for one-time inits?
          
     // if the SECOND debugPrint line is commented out the FIRST line does not print correctly, 
     // but according to the help pages (KB), the second line is not correct anyway, as sym() is required if the the symbol is specified.
        debugPrint("Script is loading1a, open(-1): " + open(-1,sym("IBM,5")) + "\n"); 
       // debugPrint("Script is loading1b, open(-1): " + open(-1,"AAPL,5") + "\n"); 
     
     // With BOTH lines I get (in order shown in debug window):
        //  Script is loading1b, open(-1): 109.64
        //  Script is loading1a, open(-1): 105.81

    // With the SECOND line commented out I get:
        //  Script is loading1b, open(-1): null

    //  Why is this null?????? (in the PREVIOUS line to the commented one). Not a good start for my first script.
    

        debugPrint("Script is loading 2\n"); 
     
     // this is what I expected to do (within an object), it doesn't work either
//     var serSym = sym("IBM" + "," + nInterval.toString()); // this is the series ofbect for this interva;
//        debugPrint("Script is loading3, open(-1): " + open(-1,serSym) + "\n"); 
 
      bStudyInit = true; 

   } 
    
   return 0;
      
} 

[eSignal_JasonK]

Hello Dave180,

The call, open(-1, sym("IBM,5")), isn't actually failing. It returns null on the first bar in the chart that gets processed because you are requesting data from the previous bar, which has not been loaded by a chart. This is occurring because your debugPrint statement is inside an initialization routine that occurs only once at the first, or oldest, bar in the chart. Once the data has been cached by the application, then it may return a valid result. The call itself will not cache the data.

To test that the call is working, try the following.

function main() {
   debugPrintln("bar " + getCurrentBarIndex() + " -- Script is loading1a, open(-1): " + open(-1,sym("IBM,5"))); 
   
   return 0;
      
} 

One other thing I should point out here is that when you request a specific bar index in a call such as the bar -1 parameter used above, you will be requesting a single value from the series. If you do not specify the bar index you can create a series object. Here's a method you can use to work with series objects.

var bStudyInit = false;

function preMain() {
    setStudyTitle("Barmy2");
    setCursorLabelName("Barmy1");
}

var xOpen = null;

function main() {

    if ( bStudyInit == false ) {
        xOpen = open(sym("IBM,5"));  // creates a series object
        bStudyInit = true;
    }
    
    var nO = xOpen.getValue(0);  // retrieve the current bar's value from the series
    if (nO == null) return;
    
    return nO;
} 

This call, open(-1,"AAPL,5"), is also legal, which is why it works. However, this method is the old EFS1 logic, which was used prior to version 7.9. The sym() function was added with the release of the EFS2 engine in that version. The advantage of sym() and inv() is that the data in the corresponding series objects are automatically synchronized by time stamp. Prior to EFS2, the synching routine between multiple intervals or symbols had to be coded manually. I recommend sticking with the use of sym() and inv() for initializing series objects.

Lastly, using an initialization flag such as bStudyInit or a BARSTATE_ALLBARS condition is simply a matter of preference.

[Dave180]

Jason, Hi

Thanks, I'm now starting to get the hang of things a bit, but I do find the absence of an integrated debug environment means it's not easy to find ones miss-understandings directly!

My work-around for this problem had been to change the series creation slightly (in the sense that I was trying to immediately confirm with debugPrint that the series I created had sensible data during this learning phase and had guessed there might be an init delay or similar):

    if (getBarState() == BARSTATE_ALLBARS) {
        debugPrint("Script is loading\n"); 

        nSymbol = getSymbol();
        nInterval = getInterval();
       debugPrint("Script is loading: "+nSymbol+", "+nInterval+"\n"); 
        return; // Prevent any series init untill we get our first NEWBAR
    }
    if (( bStudyInitAfterAllBars == false ) && (getBarState() == BARSTATE_NEWBAR)) { 
    // create series and objects here as seems you get more valid data to check
      bStudyInitAfterAllBars = true; 
} 

Is that a reasonable general approach? It seemed to help anyway.

I did notice that it is not fool-proof. For instance I created some s and ema series for different intervals, but noticed that if (as part of learning / checking) I asked for them all to be printed (each ma for each interval) for each subsequent main() invocation, some values were null (early ones) - apparently arbitrarily, and the results were variable for each reload. How can that be? I could believe it if the values were missing up until the ma length, but that's not what was happening. I printed with

this.myema08.getValue(0) 

where the series had been created along the lines of

this.serSym = sym(sSym+","+nInv.toString());
this.myema08 = ema(8,this.serSym); 

Why should the data "come and go"?

Incidentally, although your mention of the -1 and cache stuff helps, I don't see how it actually accounts for my first debug line starting to work when the second (subsequent) debug line is added (this is still bar -1 afterall), but I can live without understanding that!

Is there a document that explains the performance pros/cons of creating series (of say smas) as opposed to simply getting individual values, or open(sym("IBM,5")) vs serSym = sym("IBM,5"); and then open(n,serSym) and close(n, serSym))? My "init" period seems quite lengthy (there are 4 intervals * 4 e/smas = 16 series, plus the "sym()" one. Is that "a lot"? It may be that it is the computation of the historic bars that is part of delay, it's not clear how I can us the performance monitor on a "function" level?

You mention

The advantage of sym() and inv() is that the data in the corresponding series objects are automatically synchronized by time stamp

I also read

This will create the series that the engine requires to be able to maintain full syncronization.

and there is talk about efsInternal etc. I can't find an article that describes these synchronisation / time stamp pitfalls - where can I read about them?

Many thanks for helping me get heading in the right direction,and for carefully addressing the various queries I raised.

Cheers

[eSignal_JasonK]

Hello Dave,

You're most welcome.

A more robust debugging environment would definitely be valuable and this type of thing has been requested by others in the past. It just hasn't been an item development has had the resources to develop yet. However, feel free to submit your ideas and requests in this regard to them directly at [email protected].


The best way, in my opinion to check for valid data from a function call or series object is by performing null checks after the initialization routine. Series that require a specified number of bars of data before they are primed with their first valid result will return null until that bar has been processed. Take for example a 10-period moving average, ema(10). It won't plot a valid value until the 10th bar from the oldest bar loaded in the chart. Doing a null check is the simplest way as it doesn't require you to count any bars.

var bInit = false;
var xMA = null;

function main() {
    if (bInit == false) {
        xMA = ema(10);
        bInit = true;
    }

    var nMA = xMA.getValue(0);
    if (nMA == null) return;

    return nMA;
} 

Regarding your description of the arbitrary null returns on the subsequent invocations of main(), I'm not seeing what you're describing. Are you saying that you are getting null returns in between valid returns for a series? If so, can you post an example formula that illustrates this?

I tried the following modification of what you are doing and it seems to work just fine. In general, you should be able to assign a sym() to a variable or object property and pass that as the sym parameter for a series function such as ema(). Just make sure serSym is a series object form. One way to do that is to print it to the formula output window as I have below. You should see "[object Series]" printed.

function preMain() {
    setPriceStudy(true);
    setStudyTitle("Test");
    setCursorLabelName("ma");
}


var bInit = false;
var xMA = null;

function main() {
    if (bInit == false) {
        var sSym = getSymbol();
        var nInv = 10;
        var serSym = sym(sSym+","+nInv.toString());
        debugPrintln(serSym);
        xMA = ema(10, serSym);
        bInit = true;
    }

    var nMA = xMA.getValue(0);
    if (nMA == null) return;

    return nMA;
} 

The amount of time for the initialization of the formula is going to be more dependant on the amount of data loaded in the chart rather the number of series objects being declared. To see the difference, set up a time template to load a relatively small amount of data and run your formulas. Then compare them on a time template that loads a lot of data.

All the documentation we currently have for EFS can be found at the links below my signature. We don't have the specific documents that you are looking for but will take them into consideration for future documentation. Although I don't see a need to document time stamp synchronization. EFS2 handles this for us now, so we don't need to be concerned with lining up the data between intervals and symbols anymore. You can find more discussions about this EFS1 issue here in the forums using the search page if you like.

Regarding the retrieval of individual values or creating series, make use of individual value calls when you can. This may be a bit more efficient. However, creating the series objects is necessary when you need to create other series based on other sources, which will also need to be in a series object form. In most cases you probably need to create the series.

[Dave180]

Jason, thanks again, and apologies for the delay in responding. Had a while off EFS, but now back on it. I have a few queries about time templates and ma periods and stuff to post in another thread.

Re

Regarding the retrieval of individual values or creating series, make use of individual value calls when you can. This may be a bit more efficient. However, creating the series objects is necessary when you need to create other series based on other sources, which will also need to be in a series object form. In most cases you probably need to create the series.

I tried replacing series creation and subsquent "myVal = this.myMaSeries.getValue()" calls with direct access to values like "myVal = ema(8, this.serSym,0)" but there this seemed to reduce performance. Have I missunderstood what you meant?

Regards

Dave

[eSignal_JasonK]

Hello Dave,

Originally posted by Dave180
...

I tried replacing series creation and subsquent "myVal = this.myMaSeries.getValue()" calls with direct access to values like "myVal = ema(8, this.serSym,0)" but there this seemed to reduce performance. Have I missunderstood what you meant?

Regards

Dave

I apologize if I'm confusing you. Let me clarify. If you're not using an initialization routine assigning a single value call to a variable is more efficient that the series object assignment.

This code...

function main() {
    var n = open(sym("IBM,5"));  // series object assignment

    if (n.getValue(-1) .....
} 

is less efficient than this code ...

function main() {
    var n = open(-1, sym("IBM,5"));  // single value assignment

    if (n .....
} 

Using the initialization routine like in the examples I've posted previously in this thread is the best way to go.