5.1 Source Lines of Code
This is obtained by counting the numbers of lines of all the source
files composing the library and removing blank lines and com-
ments. This parameter should provide an indication of the effort
required to develop the application: roughly speaking, the more the
SLOC the more the time required to write the software and to test
it. However, as argued in [10], SLOC is dependent of at least three
factors: programmer’s skills, programming language and program
functionality.
Indeed, it is quite simple to understand that sources written
by different programmers, with different programming abilities,
could sensibly differ in the number of SLOC; in general, if all
programs exhibit the same functionalities, the more skilled the
programmer the less the number of lines written. In any case, given
that most of the implementations compared in this paper are part
of the library of the language we tested, we can assume that they
have been written by programmers with a very good knowledge
of the language itself and therefore they can be considered “good
software”; so, as for this point of view, it makes sense to compare
the number of SLOC of these implementations.
Another factor that affects the number of SLOC is the program-
ming language employed. Obviously, the presence of certain lan-
guage constructs has a strong impact on the lines of program used
for a certain functionality. For example, extracting the first element
of a string is obtained in Erlang by means of a simple one-line state-
ment: [H|T] = String. Moreover, if this extraction is performed
in the declaration of a function clause, as is often the case, the code
becomes even more compact, as the line will contain the function
declaration, check a condition (that the string is not empty) and per-
form a double assignment. In Python or Java, the extraction of the
first element requires two lines of code, and additional lines when
we intend to also include the “string not empty” check:
Python
Java
H = String[0]
H = string.charAt(0);
T = String[1:]
T = string.substring(1);
if String != "" :
if (string.length() != 0) {
H = String[0]
H = string.charAt(0);
T = String[1:]
T = string.substring(1);
}
This dependence of SLOC upon the language used is not only
fundamental for our study, but also quite desirable as we want
to understand the effort needed to develop the IMAP client using
different languages. The presence of certain constructs that require
less lines of code implies less time needed to write the software and
to test it.
Another important aspect that influences the number of SLOC
is the functionality the software exhibits; indeed two different im-
plementations of an application which at first glance might appear
the same but in reality differ in functionality, cannot be compared in
terms of SLOC, as the feature rich implementation will presumably
have a higher number of SLOC. This is the reason why, in evalu-
ating the software, the SLOC parameter is weighted by means of
the so-called Function Point Analysis (FPA) [8]; this is a technique
that allows developers to analyze a software implementation and
derive a parameter, called number of function points, providing an
objective evaluation of the functionalities of the analyzed software.
Given this measure, weighting the number of SLOC with respect to
the number of function points should give a more precise estimate
of the required development effort required.
However, performing FPA is not an easy task, above all when
dealing with a library. According to [7], FPA seems more appro-
priate for a complete application or software system rather than a
single component. For this reason, in the analysis of SLOC, we
used a different approach with the aim of obtaining the same ob-
jective evaluation. Indeed, as argued in Section 4, by analyzing the
implementations given with their source code (i.e. Python, Ruby,
C# and Erlang), we noted their software architectures were very
similar to one another. In particular, once the blocks for the low-
level protocol, request assembly and reply parsing had been devel-
oped, adding a new functionality (i.e. support for another IMAP
command) only involved writing a function (or method) that used
the primitives provided by these basic blocks. From such a charac-
teristic, we can derive that comparing the number of SLOC of two
implementations, even if they support different sets of commands,
can be performed by simply analyzing the parts of the software
which provide the same functionalities, namely the low-level pro-
tocol, request assembly, reply parsing and the common commands
in the different implementations.
5.2 Functionality of primitives
As argued before, Function Point Analysis is not easy to perform
and also not appropriate for a library. On the other hand, in a
comparative analysis as the one described in this paper, an under-
standing of the specific functionalities of a given library should be
mandatory. The aim should be to try to understand the quality of
an implementation in terms of its capability to provide a complete,
transparent and flexible support of the IMAP protocol.
Checking the completeness of a solution is quite simple, since
it implies to analyse if the library is able to support all or a sub-
set of the commands and functions of the IMAP4 standard. As
for transparency and flexibility, an analysis of the software archi-
tecture of the solution and the interface of the various primitives
(i.e. signatures of function or methods) is needed, in order to un-
derstand if a good level of abstraction is provided. Indeed, many
commands of the IMAP4 standard require additional parameters
which should be passed to the function (or method) implementing
that functionality. Even if they are then sent as string in the pro-
tocol messages, these parameters may vary in type and semantics,
therefore a well-written library should treat them according to their
real meaning. As an example, a function implementing the FETCH
command could require the additional parameters to be passed as
either different arguments or a complete string. In the latter case,
there is a lack of a proper abstraction level, since the programmer
has to manually create the string to be passed: in other words, to
use the library the programmer has to possess a certain knowledge
of protocol messages, an aspect which is in contrast to the common
rules of software engineering that instead require library/software
modules to hide specific low-level (protocol) details by providing a
high-level flexible and uniform interface.
A similar argumentation can be given for function/method re-
sponse values, which should reflect the outcome and the reply of
the command implemented. If a reply is already interpreted by the
library and provided as a structured data type of the language (prim-
itive or derived), the programmer can directly use it without writing
additional parsing code. Once again, this is an indicator of a proper
encapsulation and abstraction level of the library, characteristics not
featured by e.g. an implementation returning raw protocol replies,
since, in this case, an additional programming effort is required to
properly understand and use them.
5.3 Memory Consumption
Memory consumption is a performance parameter that expresses
the memory usage of an application using the IMAP library be-
longing to a certain language. We developed a simple test program
that logs in and fetches a bunch of messages. During the execution
of the test program on a Linux OS, we collected information about
memory usage by checking the status contained in the “/proc” file
system, in particular, by looking at total program size, code size
and data size.
33