ESPL: Power Programmer, Part III

In the prior two parts of this series, I covered some basic concepts about ESPL, then discussed many of the programming features present in ESPL. This segment will examine memory storage issues, and then I have a long wish list of features I’d like to see added to ESPL.

Memory Modes

Information in any computer program needs to be stored in memory, so what’s the big deal? There are two issues:

to what instances is the data available;
at point in time does the data cease to exist (referred to as persistence); and
to what part of the program is the data available (referred to as scope).

Before addressing these issues, we need to remember how Ensign Windows uses our ESPL code. Referring back to the first article in this series, our ESPL code is invoked from Ensign Windows on each occurrence of the related event. That is, an ESPL study is called for each new price tick (or at the close of each price bar, if so configured). For timer related ESPL, the code is called each time the configured time expires. For keyboard related code, it is called when the related keystroke is hit. Also remember that for studies, color bar studies and drawing tools, there is a separate instance of the ESPL program, for each study that is applied to a chart.

Instance Global, Persistent, Method Global

Since there may be many instances of our ESPL study, active at a given time, there might be data that we want to exist and be available to all instances of the study, and across invocations. That is, if we set a value in one invocation of one instance of the study, and we want that value to be available to another instance (obviously in another invocation), then we can use ESPL global data. This data is declared in the var block of the outer most var-begin-end block of code.

Instance Private, Persistent, Method Global

This next memory mode is the most important one, to me. I frequently use data that I want available only to the current instance of the study, and I want it to exist across invocations. The idea here is that each instance of a study will have it’s own set of data. The data in one instance is not to be shared with other instances. However on each invocation of an instance (i.e. each tick or closed bar), I need the values that were set on the prior invocation to be in tact, just as I left them, at the end of the prior invocation.

If you are just learning the ESPL language, you might think that ESPL does not provide for such an important memory mode. Then you take a closer look at the documentation for the SetUser() method. Here you see that Ensign provides six arrays of data, for every instance of a study. Note that the elements in these arrays are of type real, so they can only hold numbers. Boolean, String and other data types can not be stored in these arrays. I should point out that I am discussing details about the new version Ensign (aka Super Ensign). The older version has similar characteristics with some differences. Of the six arrays, the first four are available to display lines (and/or markers) on the chart. These arrays are dimensioned from 1, to the configured maximum number of bars for the chart. The fifth array is dimensioned the same. The sixth array is dimensioned from 1 to 15.

Since only the first four arrays are used to display lines on the chart, what are the fifth & sixth arrays used for? You guessed it, they are available for Private Persistent data. Any value stored in these arrays will be private to the study instance that owns the array. In addition, those values will be saved and remain available on each invocation of that instance.

To be complete about it, Ensign allows for the storage of 3 Private Persistent String data elements, with the SetUser() built-in function. This is a very welcome addition to Ensign.

Instance Private, Non-Persistent, Method Global

Next, consider the data that we want accessible to all methods (generic term for procedures and functions). It does not need to exist across invocations, nor do we want its content shared with other instances of the study. Since we don’t care what happens to the data when we leave the current invocation, then this type of data can be implemented with global data (outer most var section) also. However, it is imperative that at the start of our code, for each invocation, we initialize the data, since we have no idea what might be left there from some prior invocation or from some other instance.

Instance Private, Non-Persistent, Method Local

The final memory mode supported by ESPL is local memory. This is memory that is declared inside of a procedure or function. It is only accessible from inside that method, and it ceases to exist when execution leaves the method. This type of memory is frequently called automatic, since it automatically ceases to exist upon leaving the method. These data items are declared in the var section of each method that requires them. It is important to note that upon entry to a method, these data items have an undefined content; and so it is again imperative that your code initialize their content.

Other Memory Modes

I have used three characteristics to describe these memory modes: Instance private or global, Persistent or not, and Method global or local. With three characteristics, there should be 8 different possibilities; yet I have only discussed four. What about the rest? Most of them make no sense; however there is one that I should mention. Most programming languages allow for the dynamic (at run time) allocation of data. This data would be private to the instance that allocated it. The data is persistent, in that it survives across invocations; and yet it’s access is local to the code that knows where to find it. This last characteristic implies that we need another data item, that holds the location of the dynamically allocated data. This location holder is called a pointer. The pointer can be stored using any of the data modes already discussed. Problems arise when the pointer ceases to exist and we have not de-allocated the dynamically allocated memory. I should mention that there are ways to deal with that problem, such as is done in the Java programming language, where no explicit de-allocation is required. ESPL does not support this memory mode.

(Editor's Comment: Any number of real arrays can be created by declaring variables of type TArray. Any number of string arrays can be created by using the powerful TStringList data type. ESPL manages one global TStringList variable called sList that is global and freed by Ensign upon program termination.)

My Wish List

If we were talking about some other product, writing and distributing a wish list would most likely be a waste of time. But we are talking about Ensign Windows and Howard Arrington. Ensign (especially Super Ensign) is so full of wonderful features, but the number one thing, in my mind, that puts it way ahead of the rest, is the fantastic support provided by Howard, Kimball Hansen, Sheldon Lamont and Mike Lamont. As for new features, and wish lists, we need to realize that Howard and team, only have two hands, and 24 hours each day. Yet, when there is a problem, or a significant number of requests for a feature, then amazingly it’s in the next release, which you can download the next or following day.

Operating Environment

Multiple Code Files – As was discussed in the first article in this series, Ensign works with at most one ESPL code file, at a time. Thus if you have three ESPL studies that you like to use, you need to put them all into one ESPL file (or at least, reference other code files with the Include statement, from one master file). If you have your three studies, and you are not constantly added and changing them, then this system would work for you. On the other hand, if you play around with 10 or 20 studies, color bar studies, drawing tools, and timer or keyboard related code, then this mode of doing things becomes very cumbersome. I would like to see Ensign use multiple code files. I would like to put one study into its own file. Then tell Ensign to apply that file to the current chart. I could add as many ESPL studies to the chart as I wish, each in their own file. The same ESPL study/file could be added to multiple charts as well. To maintain the ability to have related studies together in one file, I would keep the concept of the ESPL (aka Who) value. This value would be unique within each ESPL file, but does not need to be unique across ESPL files.

More Advanced User Study Parameters – The primary method of passing configuration information to a running ESPL study, from Ensign, is through the three parameter fields in the properties window for the ESPL study. This is somewhat limiting in that:

there are only three parameters,
the parameters must be numeric, and
the names of the parameters can not be set by the ESPL code.

(Editor's Comment: The Grid Line control could be used to pass 19 Boolean check box states. The draw line color boxes can be used to pass color parameters.)

I would like to see a mechanism where the ESPL code could tell Ensign the quantity, identity (names) and type of the parameters to be shown in the study’s properties window. This could be done either through a special section in the source code file, that is read by Ensign (statically, not at run time); or it could be done through methods that the ESPL calls at run time.

A True Debugger – When testing and debugging ESPL, the only debug tool available is the writeln() method. This method writes values to the output window of the ESPL editor. I would like to see a full featured debugger, with:

break points,
source level statement stepping,
execution continuation,
variable viewing (click on a variable name to see its value, or a watch window), and
variable setting (ability to change the value of a variable, during execution).

(Editor's Comment: A variable watch window is available, but it is of limited use because of the lack of break point capability.)

In all honesty, I have no idea if any of this is even possible, with the way Ensign is written; but remember, this is a wish list.

Standard Programming Language Features

Return Statement – as I mentioned in the prior article in this series, the lack of this statement leads to an excessive use of either if-then-else statements, or goto statements. In conjunction, I would like to see the built-in variable Result replaced with the use of a parameter to a return statement.

Break and Continue Statements – these statements provide the ability to leave a looping statement (such as while, until or for), or to go to the top of the loop without completing the current pass through the loop’s body. These statements are common in many conventional programming languages. I classify them as convenience statements.

Forward Referencing – ESPL only allows backward referencing. This means that any reference to a procedure or function must occur after that procedure or function has been declared. As a result, when programming with a standard top-down approach, the top level methods must be placed at the end of the file. In addition, extreme care must be given to the order in which methods are declared in the file. Most significantly, multiple method recursions (A calls B, and B calls A) becomes impossible. I would like to see forward referencing allowed, which would alleviate the three problems mentioned here.

Structures – I would like to see the ability to declare complex data types, such as structures. A structures is a grouping of other data types (such as some integers, some reals, some Booleans, etc.) into one group, so that the group can be referenced as a whole.

Arrays of Data Types – Ensign currently does provide for arrays of reals. However these arrays are created with special built-in methods, rather than with data declaration. The use of methods is nice, since it allows the dimensioning of the array to be determined at run time. So I would like to see the current method retained. In addition, I would like to see the ability to declare arrays of any data type, including arrays of structures and arrays of arrays.

Classes and User Defined Objects – Many conventional programming languages today use a style of programming called "object oriented". Examples of such languages are C++ and Java. A class is similar to a structure, in that, it is a grouping of simpler elements; however a class also allows the defining of methods within the context of the class. Objects are instances of a class. Typically, objects are instantiated (allocated) at run time, and the methods know which object to operate on, by referencing the object and method together.

Pointers and References – a pointer is a data variable that provides a reference (i.e. an address) to some other variable. Pointers are essential when using dynamic memory allocation (discussed above, under Memory Modes). Pointers can also be used to reference statically declared data.

Preprocessor Macros – preprocessor macros allow the ESPL software developer to declare names for constant values that will be used in the code. An example might be

#define MacdValue 1

This declares the name MacdValue to be replace by the constant integer 1 whenever that name appears in the code. This happens before the code is compiled. The name is not a variable, and thus takes up no space in the running program. There is also no execution time spent, to obtain the value 1, since it was simply a constant, at the time the code was compiled. The benefit of preprocessor macros is that the code is easier to read and debug, since constant values can be replaced with meaningful names.

Charting Package Programming Language Features

Ability to download extensive historical data – when back-testing a programmed trading system, the more data it is tested with, the more reliable the system. Related to this, is the feature to be able to build charts of any timeframe, from downloaded data. Currently Ensign needs to download back data for each timeframe chart used.

(Editor's Comment: The File | File Manager | Merge tool can be used to build any time frame that is a multiple of a lower time frame data set that is present. Up to 60 days of intra-day data and 10 years of daily data can be downloaded from the eSignal servers, or by using the Internet Services download tool. Ensign data files can store up to 65,535 bars.)

More Built-in Reporting and Support for Back-Testing of Systems – The current reporting mechanism for back-testing is the Trading Details report. This report is wonderful. It reports trades taken by the ESPL program, number of winners & losers, each trade’s profit/loss, a running balance, and more. Other analysis would be useful. Such as daily and weekly trade results, maximum run up and maximum draw down. Trade results for different times of the day would also be of interest.

Built-in Support for Optimization – Optimization is the process of fine tuning a trading system, to produce the best results. This could be done by having Ensign vary the values of parameters to an ESPL trading system, over defined ranges, and by a defined step sizes. For each value, the trading system would be run, and the trading results captured. A specific value in the trading results would be specified to be the one compared for best results. When the system had been run for each of the parameter settings, an optimization report would show the trading results for each setting, and those settings that produced the best results.

In Conclusion

I am aware that the above wish list is not only lengthy, but it would seem that I am asking for the world on a silver platter. I know. But isn’t that what a wish list is for? Do not get me wrong, as I mentioned at the start of this series of articles, ESPL is the best charting system programming language I have seen. Yet even the best, can be better.

I hope that through these articles I have provided some insight into why I feel ESPL is so wonderful. I also hope that I have provided a hint or two, of things you might want to do with ESPL. I thank you for reading, and I thank Howard for giving me this opportunity to express my views.

Happy coding.