The Managed Resource Interface: Anaysis
Erlang Training and Consulting |Table of Contents|Introduction|Choice of Management Protocol|System Flexibility Requirements|The Managed Resource Interface|Management Protocol Mapping|Analysis|Conclusion| Bibliography and References|Acknowledgments|Appendix A: Erlang|Appendix B: MRI Example|Appendix C: Abbreviations

This section analysis the MRI. It looks at how it can interface in with the standardised management protocols discussed in this thesis, and how it can be used on a stand alone basis. It concludes by looking at some of the problems which were experienced in the existing implementations.


The Managed Resource Interface was first implemented in the ANX DSL project as a means of removing the SNMP dependencies in the logical appplications of the CP, facilitatating the insertion of new standardised management protocols. This was a major rewrite of the existing system, the ANX HFC, involving several geographically separated teams of engineers. The MRI was later used in the AxM3 project, with the MRI going into the design of the system from the start. In both these projects, SNMP and HTTP were interfaced towards the MRI. The analysis in this section is based on these project experiences.


The Coexistence of the MRI

Standardized management protocols differ in terms of power. Simple protocols such as HTTP or SNMP result in simple agents but complex applications. On the other end of the scale, complex protocols such as CMIP and Corba have complex agents but simple applications. It is a trade off on where the complexity is placed, namely in the data definition of the information model or in the functions available to access and manipulate this data. The Management Resource Interface can be placed in the middle of the discussed protocols, both in terms of data definition and complexity of the functions that can be applied on this data.

SNMP’s information model consists of a complex information model with simple functions to manipulate it. This pattern is even more evident in the case of HTTP. Corba and CMIP, on the other hand, have simple information models, but more complex functions to access and manipulate the data, making the protocols more powerful.

This means that the mapping from SNMP and HTTP to the MRI is a simple task. There is no functional gap that has to be filled, as the functions in the MRI are more powerful. There is only a semantic gap mapping the HTTP and SNMP requests to MRI records and operations.

When mapping Corba and CMIP, however, functionality not available in the MRI has to be implemented. This functional gap includes transaction, event, notification and time services, all of which exist in the Erlang/OTP Corba agent. This mapping can easily be implemented by inserting a bridge that handles the functional gap. The semantic translation can then be placed as a layer between the bridge and the MRI, or as shown in diagram 5, right above the MRI. If such a bridge was needed (It depends on what Corba services are used), with a little care it could be implemented generically allowing its re-usage among projects. No Erlang – CMIP implementation exists today, but a bridge would follow a similar principle as the Corba one, only that it would be much more complex, as there is a larger functional gap which has to be filled.


The MRI as a Proprietary Protocol

If an Element Manager is implemented in Erlang, the MRI should be used as a proprietary management protocol. Using communication protocols other than distributed Erlang adds an unnecessary factor of complexity to the software, penalizing the performance due to overheads such as the encoding and decoding of data.

If distributed Erlang is used, an extra layer of functionality should be included above the MRI. Issues such as node monitoring, and network optimizations, user authentication and encryption should be covered in this layer. This internal communication system should also work as a wrapper and a single point of entry to the NE, ensuring that the protocol is followed. Quick "work arounds" such as direct calls to the MRI should not be done, as testing, debugging and maintenance will become harder.

Security is not an issue if the program is executing in a closed network configured solely for that purpose [16], but such might not always be the case. Applications might want to access an Erlang system through an open network.

It is very questionable if the security mechanisms that exist in distributed Erlang are acceptable to run applications on open networks. By configuring the net kernel, message passing among nodes could be limited to a registered process. This will stop all processes on other nodes from executing remote procedure calls, spawning processes, or executing any other operation allowed in a distributed Erlang environment.

Message passing, remote procedure calls, and all other inter-node communication is not encrypted. A secure socket layer library allows data exchanged through port communication to be encrypted. Using ports between Erlang nodes, however, poses limitations and hides the transparency provided by the Erlang distribution described in this section. Another option would be the crypto application, which encodes and decodes Erlang terms. Performance would in both cases be affected. Several research papers have been written suggesting how Distributed Erlang can be made more secure. For ongoing work on safe Erlang, see [Brown, 2000].


Generic Utilities

Generic tools can easily be implemented and reused across projects. These tools can be seen as agents and bridges, simplifying the mapping from standardized management protocols, providing services not handled in the MRI. They can also act as caches and proxies, handling local optimizations.

Bridging and Interfacing

The get_next operation is the key operation in SNMP. It was not included in the MRI as it is too closely related to SNMP, and not implemented in other standardized protocols. Implementing a generic function handling the get_next request on all MRI tables, however, would not only allow the re-usage of code, but it would also allow local optimizations.

In order to implement the get_next operation, three steps are required. The first is to call the get_keys function. The list it returns is scanned for a lexicographically larger key which is used in the get_info call for that managed object. The get_keys is an expensive operation, as it includes a match on a table and the displacement of a lot of memory to transfer the keys. The match operation will affect the real time properties of the system as it is implemented using a BIF, while the memory displacement might in turn trigger the garbage collector. An optimal solution would be to cache the keys, and refresh them based on some predefined algorithm. When using SNMP, there is no guarantee that the state of the managed entities displayed in the EM is up to date with the one stored in the CP, so caching would greatly improve performance with no risk for further inconsistencies.

The get_next operation could be generalized even more and be called get_more. Passing the number of MOs one wants to retrieve to the get_more would return a list of MRI records, allowing for the efficient implementation of the GET-BULK request in SNMP. This reasoning goes further and includes the ability to filter on certain parameters in the MRI record keys or any of its fields.

Generic Tools

Generic subsystems can be written to work with the MRI, and be reused in different applications. These subsystems could include generic software upgrade routines on a device processor level, alarm filtering and suppression, as well as performance monitoring. The MRI provided the exact division between the logical parts of the system and the physical one, greatly facilitating the generic aspects of the implementation.

One of the tools implemented in the AXM3 project was a configuration manager that created and used system configuration files based on MRI actions. By starting a central processor and a management system, an operator could use the logic of the existing software to configure the system. A subsystem in the MRI monitored all the MRI actions, and if the execution was successful, it logged it on a file. An action was successful, if all the range and syntax checks in the SNMP agent were successful alongside all the logical checks in the CP. The result was a machine made syntactically and semantically correct file, as it was the same logic that checked configurations on a live system. This file could then be read and played on a real system, configuring it upon initial starts or during runtimewhen extra hardware nodes are added.


Design Flaws

MRI operations

The get_keys function proved to work well as long as the tables had less than a couple hundred elements. As soon as the table got larger, retrieving the list of keys, sorting it and forwarding it to the process executing the request caused bursts of memory usage. In the ANx project, these problems appeared on the cross connection table, where the get_keys function had to be executed often in order to retrieve small subsets of keys. This problem was solved by allowing filtering on certain tables. The filtering, unfortunately, was implemented by inserting a new function in the MRI. This function was not standardized and implemented everywhere.

The syntax of the get_keys function had originally been defined to allow a future expansion with filtering on the keys. It takes an empty record as an argument, so by setting fields in this record, filtering functionality could have been added uniformly with little impact on the protocol. The functionality was never implemented in the first version of the MRI as its need was not certain. Simplicity was chosen over complexity, leaving the option open.

Matching on table keys is efficiently implemented in ets tables. Had that not been enough, a study on how the matching on fields other than the keys would affect the design. This was originally never an option, as it would have greatly increased the complexity of the CP code under the MRI, not allowing the contents of a MRI record to be spread across several tables. Other limitations included issues such as filtering on data retrieved from hardware in real time. It is also possible that local optimizations could have been achieved above the MRI, solving this problem, and it is unfortunate that neither approach was taken.

Erlang

A major problem in Erlang has already been covered, namely the lack of efficient low-level ets operations allowing the retrieval of the table in chucks. In the R7 OTP release, ordered ets tables were implemented. This means that for large tables, the sort operation is not necessary any more when retrieving all the keys. In OTP R8, an additional match function returning a specified number of elements is available for every call. This would have been a perfect solution, but was only available through the use of mnesia queries and cursors, an operation which was too CPU intensive for a soft real time system. None of these options were available at the time.

A problem remaining, however, is the lack of functionality available for the generic manipulation of records. This lack of functionality makes it very hard to implement generic reusable code for record manipulation. In the AxM3 project, a completely generic MRI core was implemented using callbacks. This, unfortunately, was done using the tuple implementation of records. BIFS to generically create, update and retrieve data from records are needed to take full advantage of Erlang’s dynamic type system.

A full set of required functions follow. They can be easily implemented in the preprocessor and inserted in modules using the include files where the records are defined.

  • create_record(RecordType) => #RecordType{}
  • set_record_field(Record, Field, FieldValue) -> NewRecord
  • get_record_field(Record, Field) -> FiledValue
  • record_fields(RecordType) -> [Field1, .., FieldN]
  • record_type(Record) -> RecordType

Apart from these two problems, Erlang/OTP was once again proved to be an excellent tool in the development of distributed soft real time control systems.

 

[16] For example, a closed network managing a telecom application.