Page 1
1
Productivity Gains in Erlang
Productivity Gains in Erlang
A Heriot
Heriot-Watt and Motorola Production
Watt and Motorola Production
Commercial Users of Functional Programming
Commercial Users of Functional Programming
Freiburg, Germany,
Freiburg, Germany, October
October 4
th
th
2007
2007
Jan Henry Nyström
jann@macs.hw.ac.uk
jan@erlang-consulting.com
Phil
Phil Trinder
Trinder
trinder@macs.hw.ac.uk
trinder@macs.hw.ac.uk
David King
David King
david.king@praxis
david.king@praxis-his.com
his.com
Page 1
Project
Project
Project
Project
High
High-Level Techniques for Distributed Telecoms
Level Techniques for Distributed Telecoms
Software
Software
EPSRC (UK Govt) Project, Dec 2002
EPSRC (UK Govt) Project, Dec 2002 – Feb 2006
Feb 2006
Collaboration between
Collaboration between
– Motorola UK Labs
– Heriot-Watt University

Page 2
2
Page 2
Project
Project
High
High-Level Techniques for Distributed Telecoms
Level Techniques for Distributed Telecoms
Software
Software
EPSRC (UK Govt) Project, Dec 2002
EPSRC (UK Govt) Project, Dec 2002 – Feb 2006
Feb 2006
Collaboration between
Collaboration between
– Motorola UK Labs
– Heriot-Watt University
Aim:
Aim: Produce scientific evidence that high
Produce scientific evidence that high-level
level
distributed languages like Erlang or Glasgow
distributed languages like Erlang or Glasgow
distributed Haskell (
distributed Haskell (GdH
GdH) can improve distributed
) can improve distributed
software robustness and productivity.
software robustness and productivity.
Page 3
Erlang Comparisons
Erlang Comparisons
A number of sequential comparisons
A number of sequential comparisons
– Computer Language Shootout
Very few distributed system comparisons published!
Very few distributed system comparisons published!
Ulf Wiger [Wiger01] reports
Ulf Wiger [Wiger01] reports
– Erlang systems have between 4 and 10 times less code than
C/C++/Java/PLEX systems
– Similar error rates/line of code, similar productivity rates
– No direct comparative measurements

Page 3
3
Page 4
Research Questions: Potential Benefits
Research Questions: Potential Benefits
RQ1: Can robust, configurable systems be readily
RQ1: Can robust, configurable systems be readily
developed?
developed?
Resilience to extreme loads
Resilience to extreme loads
Availability in the face of hardware & software failures
Availability in the face of hardware & software failures
Dynamic
Dynamic reconfigurability
reconfigurability on available hardware
on available hardware
RQ2: Can productivity and maintainability be improved?
RQ2: Can productivity and maintainability be improved?
How do the sizes of the C++ and Erlang components
How do the sizes of the C++ and Erlang components
compare & what language features contribute to size
compare & what language features contribute to size
differential?
differential?
Page 5
Research Questions: Feasibility
Research Questions: Feasibility
High
High-level distributed languages:
level distributed languages:
Abrogate control of low
Abrogate control of low-level coordination aspects, so
level coordination aspects, so
RQ3: can the required functionality be specified?
typically pay space and time penalties for their
typically pay space and time penalties for their
automatic coordination management.
automatic coordination management.
RQ4: can acceptable performance be achieved?
RQ5: What are the costs of interoperating with
RQ5: What are the costs of interoperating with
conventional technology?
conventional technology?
RQ6: Is the technology practical?
RQ6: Is the technology practical?

Page 4
4
Page 6
Research Strategy
Research Strategy
Reengineer telecoms applications in
Reengineer telecoms applications in GdH
GdH and Erlang
and Erlang
– Dispatch Call Controller [NTK04,NTK05]
– Data Mobility component
Compare high
Compare high-level and Java/C++ implementations for
level and Java/C++ implementations for
– Performance
– Robustness
– Productivity
– Impact of programming language constructs
Page 7
First Product Application
First Product Application
Communicates with Motorola mobile devices
Communicates with Motorola mobile devices
3000 lines of C++
3000 lines of C++
Uses 18,000 lines of Motorola C library functions
Uses 18,000 lines of Motorola C library functions
Has a transmitter, a receiver and 2 message types
Has a transmitter, a receiver and 2 message types
Interacts with 5 other components of the system
Interacts with 5 other components of the system
Data Mobility Component (DM)
Data Mobility Component (DM)

Page 5
5
Page 8
Second Application
Second Application
Handles mobile phone calls
Handles mobile phone calls
A process manages each call
A process manages each call
Scalable with multiple servers
Scalable with multiple servers
Despatch Call Controller (DCC)
Despatch Call Controller (DCC)
Page 9
Two Erlang DM Implementations
Two Erlang DM Implementations
1.1. Pure Erlang DM
Pure Erlang DM
2.2. Erlang / C DM
Erlang / C DM
– Reuses some C DM libraries & drivers
Both interoperate with a test harness
Both interoperate with a test harness
Combines
Combines
– Unix Processes
– Erlang processes
– C Threads (Erlang/C DM)

Page 6
6
Page 10
RQ3 Performance 1: Throughput
RQ3 Performance 1: Throughput
• Maximum DM Throughput at 100% QoS
• Pure Erlang DM is twice as fast as C++ DM
• Better process management and marshalling
• Erlang/C DM is ½ speed of C++ DM
• Still meets nominal throughput
940
940
230
230
480
480
Pure Erlang DM
Pure Erlang DM
Erlang/C DM
Erlang/C DM
C++ DM
C++ DM
Platform: 167MHz, 128Mb Sun Ultra 1, SunOS 5.8
Page 11
RQ3 Performance 2: Roundtrip Times
RQ3 Performance 2: Roundtrip Times
0
1
2
3
4
5
6
7
Query type 1
Query type 2
Broken query
C++ DM
Erlang/C DM
Pure Erlang DM
ms
• Pure Erlang is approximately 3 times faster
• Erlang/C is 26% - 50% slower

Page 7
7
Page 12
Performance Analysis
Performance Analysis
Pure Erlang DM is faster due to fast lightweight process
Pure Erlang DM is faster due to fast lightweight process
management
management
Erlang/C is slower due to additional communication to
Erlang/C is slower due to additional communication to
C components
C components
Page 13
0
1000
2000
3000
4000
5000
6000
7000
C++ DM
Erlang/C DM
Erlang DM
K
b
ERTS
Moto C lib
App C/C++
App Erlang
Performance 3: Memory Residence
Performance 3: Memory Residence
Erlang
Erlang DMs
DMs use
use 170% more memory
170% more memory
Erlang runtime sys (ERTS) has a fixed size
Erlang runtime sys (ERTS) has a fixed size
Would be a smaller % of a larger application
Would be a smaller % of a larger application

Page 8
8
Page 14
0
200
400
600
800
1000
1200
2
4
0
3
1
0
4
8
0
9
4
0
1
4
0
0
1
9
0
0
4
7
0
0
9
4
0
0
1
6
0
0
0
2
5
0
0
0
4
8
0
3
1
0
2
4
0
Erlang A
C++ A
Pure Erlang A
Load (queries/s)
Throughput
(queries/s)
RQ1 Robustness 1: Resilience
RQ1 Robustness 1: Resilience
Page 15
When overloaded:
When overloaded:
C++ DM fails catastrophically
C++ DM fails catastrophically
Pure Erlang & Erlang/C
Pure Erlang & Erlang/C DMs
DMs:
Throughput degrades
Throughput degrades
– Never completely fails
– Handling 48 q/s at peak load (25000q/s)
– Recovers automatically after load drops
RQ1 Robustness 1: Resilience
RQ1 Robustness 1: Resilience

Page 9
9
Page 16
Robustness 2: Availability
Robustness 2: Availability
Erlang DCC system:
Erlang DCC system:
Remains available despite
Remains available despite
repeated hardware and
repeated hardware and
software failures
software failures
Performance doesn’t
Performance doesn’t
degrade with repeated
degrade with repeated
failures
failures
DCC Throughput with Repeated Failures
Page 17
Robustness 2: Availability
Robustness 2: Availability
5-processor DCC with Multiple Failures
Erlang DCC system:
Erlang DCC system:
Resists the simultaneous
Resists the simultaneous
failure of multiple
failure of multiple
components
components
When more components
When more components
fail throughput drops lower
fail throughput drops lower
& recovery takes longer
& recovery takes longer

Page 10
10
Page 18
Erlang DCC system:
Erlang DCC system:
Dynamically adapt to the
Dynamically adapt to the
available hardware
available hardware
resources
resources
5 processor system
5 processor system
– Remove a processor 4 times
– Add a processor 4 times
Robustness 3: Dynamic Configurability
Robustness 3: Dynamic Configurability
DCC Throughput
with Varying Numbers of Processors
Page 19
• Shorter programs are
• Faster to develop
• Contain fewer errors [Wiger01]
• Easier to maintain
• The metric we use is Source Lines Of Code (SLOC)
RQ2: Productivity & Maintainability
RQ2: Productivity & Maintainability

Page 11
11
Page 20
398
398
398
398
Erlang
Erlang
863
863
616
616
247
247
Erl
Erl./C
./C
3101
3101
3101
3101
C++
C++
Total
Total
Erlang
Erlang
C/C++
C/C++
Lang.
Lang.
4882
4882
4882
4882
Erlang
Erlang
14.9K
14.9K
83
83
14.8K
14.8K
C++
C++
Total
Total
Erlang
Erlang
IDL
IDL
C++
C++
Lang.
Lang.
DM Implementations
DCC Implementations
•Erlang DCC & DM are less than 1/3rd of size of C++ version
•Consistent with Wiger & folklore
Productivity: Source Code Sizes
Productivity: Source Code Sizes
Page 21
0
500
1000
1500
2000
2500
3000
3500
C++ DM
Erlang/C DM
Pure Erlang DM
C++/C
Erlang
Erlang/C DM is 1/3rd of the size of the C++ DM
Erlang/C DM is 1/3rd of the size of the C++ DM
Pure Erlang DM is 1/7th of the size of the C++ DM
Pure Erlang DM is 1/7th of the size of the C++ DM
Erlang/C DM is 1/18th of the size of the C++ DM + libraries
Erlang/C DM is 1/18th of the size of the C++ DM + libraries
Productivity: DM Source Code Sizes
Productivity: DM Source Code Sizes

Page 12
12
Page 22
Reasons for difference in Code Size
Reasons for difference in Code Size
• Erlang programmers can
– Rely on fault tolerance and code for the successful case
– 27% of C++ DM code is defensive
• And have to their use
– Automatic memory management (11% of C++ DM code)
– High-level communication (23% of C++ DM code)
– Telecom design pattern libraries
Page 23
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
C++ A Moto Clib Erlang/C
Erlang
Defensive
Defines
Includes
Type Delcarations
Communication
Memory Management
Process Management
App
DM Code Breakdown
DM Code Breakdown

Page 13
13
Page 24
void DataMobilityRxProcessor: processUnsupVer(void)
{
MSG_PTR
msg_buf_ptr;
MM_DEVICE_INFO_MSG *msg_ptr;
RETURN_STATUS
ret_status;
UINT16
msg_size;
/ Determine size of ici message
msg_size = sizeof( MM_DEVICE_INFO_MSG);
/ Create ICI message object to send to DMTX so it sends a Device Info
/
message to VLR and HLR clients
IciMsg ici_msg_object( MM_DEVICE_INFO_OPC, ICI_DMTX_TASK_ID, msg_size);
/ Retrieve ICI message buf er pointer
msg_buf_ptr = ici_msg_object.getIciMsgBufPtr();
/ Typecast pointer from (void *) to (MM_DEVICE_INFO_MSG *)
msg_ptr = (MM_DEVICE_INFO_MSG *)msg_buf_ptr;
/ Populate message buf er
SET_MM_DEVICE_INFO_DEVICE_TYPE( msg_ptr, SERVER);
SET_MM_DEVICE_INFO_NUM_VER_SUPPORTED( msg_ptr, NUM_VER_SUPPORTED);
SET_MM_DEVICE_INFO_FIRST_SUP_PROTO_VERS( msg_ptr, PROTO_VERSION_ONE);
/ Send message to the DMTX task
ret_status = m_ici_io_ptr->send(&ici_msg_object);
/ Check that message was sent successful y
if (ret_status != SUCCESS)
{
/ Report problem when sending ICI message
sz_er _msg( MAJOR, SZ_ERR_MSG_ERR_OPCODE, __FILE__, __LINE__,
"DataMobilityRxProcessor processUnsupVer: failure sending "
" device info message to DMTX");
}
Code Difference Example
Code Difference Example
Erlang
sz_dme_dmtx:cast(device_info)
C++
Page 25
Erlang DCC Reusability
Erlang DCC Reusability
36%
36%
11
11
1741
1741
Testing &
Testing &
Statistics
Statistics
3%
3%
1
147
147
Specific
Specific
Services
Services
61%
61%
26
26
2994
2994
Reusable
Reusable
Platform
Platform
Percentage
Percentage
No. Modules
No. Modules
SLOC
SLOC
Part
Part
•Considerable potential for reuse

Page 14
14
Page 26
Summary
Summary
Investigated high
Investigated high-level distributed language
level distributed language
technology for telecoms software
technology for telecoms software
Reengineered two telecoms components in Erlang
Reengineered two telecoms components in Erlang
Measured & compared the Erlang & C++ components
Measured & compared the Erlang & C++ components
Page 27
Improved resilience:
Improved resilience:
– Erlang DM and DCC sustain throughput at extreme loads
– Automatically recover when load drops
– C++ DM fails catastrophically (predict C++/CORBA DCC would)
Improved availability:
Improved availability:
– Erlang DCC recovers from repeated & multiple failures
– Predict C++/CORBA DCC would fail catastrophically
Dynamic Reconfiguration
Dynamic Reconfiguration
– Erlang DCC can be dynamically reconfigured to available hardware
– C++/CORBA DCC can also be dynamically reconfigured using CORBA
Potential for hot
Potential for hot-code loading (not demonstrated)
code loading (not demonstrated)
RQ1:Robust and Configurable Systems
RQ1:Robust and Configurable Systems

Page 15
15
Page 28
Erlang DM and DCC:
Erlang DM and DCC:
– Less than 1/3
rd
size of C++ implementation
– Erlang DM 1/18
th
size of C++ DM with libraries
– Good reusability
Reasons:
Reasons:
– Code for successful case – saves 27%
– Automatic memory management – saves 11%
– High-level communications – saves 23%
– Telecom design pattern libraries
RQ2:Productivity and Maintainability
RQ2:Productivity and Maintainability
Page 29
Even though Erlang abstracts over low
Even though Erlang abstracts over low-level
level
coordination, the required DM and DCC functionality
coordination, the required DM and DCC functionality
is readily specified.
is readily specified.
Distributed Functionality
Distributed Functionality

Page 16
16
Page 30
Time:
Time:
Max. throughput at 100%
Max. throughput at 100% QoS
QoS:
– Pure Erlang DM is twice as fast as C++ DM
– Erlang/C is ½ as fast as C++ DM , but still exceeds throughput
requirements
Roundtrip times
Roundtrip times
– Pure Erlang DM is three times as fast as C++ DM
– Erlang/C is between 26% and 50% slower as C++ DM
Space:
Space:
Pure Erlang and Erlang/C both have
Pure Erlang and Erlang/C both have 170% greater
170% greater
memory residency
memory residency due to (fixed size) 5Mb runtime
due to (fixed size) 5Mb runtime
system
system
RQ4: Performance
RQ4: Performance
Page 31
Erlang
Erlang DMs
DMs interoperate with a C test harness, and
interoperate with a C test harness, and
Erlang/C DM incorporates C drivers & library functions
Erlang/C DM incorporates C drivers & library functions
Costs
Costs
– Low space cost: an additional 15% residency
– High time cost:
• Erlang/C roundtrip times up to 6 times pure Erlang
• Erlang/C max. throughput ¼ of pure Erlang
Potential for incremental re
Potential for incremental re-engineering of large
engineering of large
systems
systems
RQ5: Interoperation Costs
RQ5: Interoperation Costs

Page 17
17
Page 32
Erlang is available on several HW/OS platforms,
Erlang is available on several HW/OS platforms,
including the Sun/Solaris DM product platform
including the Sun/Solaris DM product platform
Is supported with training, consultancy, libraries etc.
Is supported with training, consultancy, libraries etc.
RQ6: Pragmatics
RQ6: Pragmatics
Page 33
Erlang offers robustness & productivity benefits for
Erlang offers robustness & productivity benefits for
distributed telecoms software (
distributed telecoms software (RQs
RQs 1 & 2)
1 & 2)
High
High-level distributed languages like Erlang can
level distributed languages like Erlang can
deliver the required telecoms functionality and
deliver the required telecoms functionality and
performance (
performance (RQs
RQs 3 & 4)
3 & 4)
Erlang can interoperate with existing technologies
Erlang can interoperate with existing technologies
and meets pragmatic requirements (
and meets pragmatic requirements (RQs
RQs 5 and 6)
5 and 6)
Conclusions
Conclusions

Page 18
18
Page 34
Further Information
Further Information
Web sites/Seminars
Web sites/Seminars
Project Site: www.macs.hw.ac.uk/dsg/telecoms/
References
References
[NTK04] Nystrom J.H. Trinder P.W. King D.J. Evaluating Erlang
for Robust
Telecoms Software S3S’04, Chicago, July 2004.
[NTK05] Nystrom J.H. Trinder P.W. King D.J. Are High-Level
languages suitable for
Robust Telecoms Software? SafeComp’05, Fredrikstad, Norway,
Sept. 2005.
[Wiger01] Ulf Wiger, Workshop on Formal Design of Safety
Critical Embedded
Systems, Munich, March 2001
http://www.erlang.se/publications/Ulf_Wiger.pdf