Giter VIP home page Giter VIP logo

libnetconf2's Introduction

libnetconf2 – The NETCONF protocol library

BSD license Build Docs Coverity Codecov

libnetconf2 is a NETCONF library in C intended for building NETCONF clients and servers. NETCONF is the NETwork CONFiguration protocol introduced by IETF.

libnetconf2 is a NETCONF library in C handling NETCONF authentication and all NETCONF RPC communication both server and client-side. Note that NETCONF datastore implementation is not a part of this library. The library supports both NETCONF 1.0 (RFC 4741) as well as NETCONF 1.1 (RFC 6241). The main features include:

libnetconf2 is maintained and further developed by the Tools for Monitoring and Configuration department of CESNET. Any testing or improving/fixing the library is welcome. Please inform us about your experiences with using libnetconf2 via the issue tracker.

Besides the libyang, libnetconf2 is another basic building block for the Netopeer2 toolset. For a reference implementation of NETCONF client and server, check the Netopeer2 project.

Branches

The project uses 2 main branches master and devel. Other branches should not be cloned. In master there are files of the last official release. Any latest improvements and changes, which were tested at least briefly are found in devel. On every new release, devel is merged into master.

This means that when only stable official releases are to be used, either master can be used or specific releases downloaded. If all the latest bugfixes should be applied, devel branch is the one to be used. Note that whenever a new issue is created and it occurs on the master branch, the first response will likely be to use devel before any further provided support.

libnetconf vs libnetconf2

libnetconf2 was developed with experiences gained from the development of the libnetconf library, which is now obsolete and should not be used.

Packages

Binary RPM or DEB packages of the latest release can be built locally using apkg, look into README in the distro directory.

Requirements

  • C compiler (gcc >= 4.8.4, clang >= 3.0, ...)
  • cmake >= 3.5.0
  • crypt(3)
  • libyang
  • libssh >= 0.9.5 (for SSH support)
  • OpenSSL >= 3.0.0 (for TLS support)
  • curl >= 7.30.0

Optional

  • libpam (for PAM-based SSH keyboard-interactive authentication method)
  • libval (only for DNSSEC SSHFP retrieval)
  • doxygen (for generating documentation)
  • cmocka >= 1.0.1 (for tests only, see Tests)
  • valgrind (for enhanced testing)
  • gcov (for code coverage)
  • lcov (for code coverage)
  • genhtml (for code coverage)

Building

$ mkdir build; cd build
$ cmake ..
$ make
# make install

The library documentation can be generated directly from the source codes using Doxygen tool:

$ make doc

Build Options

There are various options to change result of building.

Changing Compiler

Set CC environment variable:

$ CC=/usr/bin/clang cmake ..

Installation Prefix

By default, the library is installed with the /usr/local prefix, to change it, use the following option:

$ cmake -DCMAKE_INSTALL_PREFIX:PATH=/usr ..

Transport Protocol Support

The NETCONF protocol specification allows to use the protocol on top of several transport protocols. libnetconf2 provides support for SSH and TLS transport. By default, both SSH and TLS transport is enabled. Disabling and enabling both the transport protocols can be made in the same way. The following command has actually the same effect as specifying no option since it specifies the default settings.

$ cmake -DENABLE_SSH_TLS=ON ..

DNSSEC SSHFP Retrieval

In SSH connections, if the remote NETCONF server supports it and it is enabled, it is possible to safely retrieve server host key fingerprints using DNSSEC and automatically consider them to be trusted without any interaction. Enable it with the following command.

$ cmake -DENABLE_DNSSEC=ON ..

Build Modes

There are two build modes:

  • Release. This generates library for the production use without any debug information.
  • Debug. This generates library with the debug information and disables optimization of the code.

The Debug mode is currently used as the default one. to switch to the Release mode, enter at the command line:

$ cmake -D CMAKE_BUILD_TYPE:String="Release" ..

Inactive Read Timeout

It is possible to adjust inactive read timeout. It is used when a new message is being read and no new data had arrived for this amount of seconds. 20 is the default value.

$ cmake -D READ_INACTIVE_TIMEOUT:String="20" ..

Active Read Timeout

Active read timeout is used to limit the maximum number of seconds a message is given to arrive in its entirety once a beginning is read. The default is 300 (5 minutes).

$ cmake -D READ_ACTIVE_TIMEOUT:String="300" ..

PSPoll Thread Count

This value limits the maximum number of threads that can concurrently access (wait for access) a single pspoll structure. To simplify, how many threads could simultaneously call a function whose parameter is one and the same pspoll structure. If using netopeer2-server, it will warn that this value needs to be adjusted if too small.

$ cmake -D MAX_PSPOLL_THREAD_COUNT:String="6" ..

Code Coverage

Based on the tests run, it is possible to generate code coverage report. But it must be enabled and these commands are needed to generate the report:

$ cmake -DENABLE_COVERAGE=ON ..
$ make
$ make coverage

Note that gcc compiler is required for this option.

CMake Notes

Note that, with CMake, if you want to change the compiler or its options after you already ran CMake, you need to clear its cache first - the most simple way to do it is to remove all content from the 'build' directory.

Usage

All public functions are available via 2 headers:

#include <nc_server.h>
#include <nc_client.h>

You need to include either one if implementing a NETCONF server or a NETCONF client, respectively.

To compile your program with libnetconf2, it is necessary to link it with it using the following linker parameters:

-lnetconf2

Examples

See examples directory for an example client and server.

Tests

The repository includes several tests built with cmocka. The tests can be found in tests subdirectory and they are designed for checking library functionality after code changes.

The tests are by default built in the Debug build mode by running

$ make

In case of the Release mode, the tests are not built by default (it requires additional dependency), but it can be enabled via cmake option:

$ cmake -DENABLE_TESTS=ON ..

Note that if the necessary cmocka headers are not present in the system include paths, tests are not available despite the build mode or cmake's options.

Tests can be run by the make's test target:

$ make test

libnetconf2's People

Contributors

adamjrichter avatar alangefe avatar apropp-molex avatar bhart3 avatar brchiu avatar dajvid avatar fanchanghu avatar fredgan avatar frimpler avatar jktjkt avatar jruzicka-nic avatar kpbarrett avatar lepici avatar mekleo avatar michalvasko avatar mohitarora24 avatar neheb avatar olivier-matz-6wind avatar pavolvican avatar petr-sokolov avatar preetbhansali avatar ravsz avatar rjarry avatar rkrejci avatar rovarga avatar roytak avatar synther avatar syyyr avatar tieliao avatar wungad avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

libnetconf2's Issues

error: "LYVE_INVALATTR" undeclared (first use in this function)

Hello,

I am trying to install libnetconf2 as part of netopeer-GUI.
I am running on RedHat7.2 and have installed netopeer-GUI successfully before on the same device.

after updating the libyang and libnetconf2 libraries and following the install steps I am facing an issue whenever I run the 'make' command in libnetconf2

I am getting the below error:

/var/www/netopeer-gui/install/netopeerguid/libnetconf2/src/messages_server.c: In function "nc_err_libyang":

/var/www/netopeer-gui/install/netopeerguid/libnetconf2/src/messages_server.c:471:14: error: "LYVE_INVALATTR" undeclared (first use in this function) case LYVE_INVALATTR:

/var/www/netopeer-gui/install/netopeerguid/libnetconf2/src/messages_server.c:471:14: note: each undeclared identifier is reported only once for each function it appears in make[2]: *** [CMakeFiles/netconf2.dir/src/messages_server.c.o] Error 1 make[1]: *** [CMakeFiles/netconf2.dir/all] Error 2 make: *** [all] Error 2

Let me know if you need any further information from my end.

Thank you for your support.

Maged

Require asynchronous mode support for NETCONF session

Hi,
This requirement is for asynchronous mode support for NETCONF session, now, client must wait for the previous response before sending a new request in one session. (the new request is blocked if the previous response is not received and sent to client.)

Whether to support "when" syntax ?

i have a yang module ,it has "when" synatx, I find "relove_when" return error, i wonder how to judge this variable(type) value ? or it does not support ?

when "/oc-platform:components/oc-platform:component/" +
"oc-platform:state/oc-platform:type = 'OPTICAL_CHANNEL'"

When condition "/openconfig-platform:components/openconfig-platform:component/openconfig-platform:state/openconfig-platform:type = 'OPTICAL_CHANNEL'" not satisfied.

TLS session cleanup (nc_session_free function) leaking the socket at SSL layer failures.

The test we did had a TLS call-home configuration where the NC client didn't accept the netopeer2-servers certificate (by mistake though), and the client side closed the SSL session. This loop was running continuously every second.
Depending on the timing, the netopeer2-servers side, can fail at any of the following functions with the given error.

nc_read()/nc_write()/nc_read_poll()
...
ERROR: Call Home client "tls_ch_client" session terminated, reconnecting...
VERBOSE: Trying to connect to 192.168.16.2.
VERBOSE: Successfully connected to 192.168.16.2:4335 over IPv4.
netopeer2-server[3900]: Resolving unresolved data nodes and their constraints...
netopeer2-server[3900]: All data nodes and constraints resolved.
ERROR: Call Home client "tls_ch_client" session 242 established.
ERROR: Session 242: communication socket unexpectedly closed (OpenSSL).
netopeer2-server[3900]: Session 242: thread 3 event session terminated.
netopeer2-server[3900]: Generated new event (netconf-session-start).
netopeer2-server[3900]: Generated new event (netconf-session-end).
ERROR: Call Home client "tls_ch_client" session terminated, reconnecting...
VERBOSE: Trying to connect to 192.168.16.2.
VERBOSE: Successfully connected to 192.168.16.2:4335 over IPv4.
....
VERBOSE: Trying to connect to 192.168.16.2.
VERBOSE: Successfully connected to 192.168.16.2:4335 over IPv4.
netopeer2-server[3900]: Resolving unresolved data nodes and their constraints...
netopeer2-server[3900]: All data nodes and constraints resolved.
ERROR: Call Home client "tls_ch_client" session 325 established.
ERROR: Session 325: communication socket unexpectedly closed (OpenSSL).
netopeer2-server[3900]: Session 325: thread 2 event session terminated.
netopeer2-server[3900]: Generated new event (netconf-session-start).
netopeer2-server[3900]: Generated new event (netconf-session-end).
ERROR: Call Home client "tls_ch_client" session terminated, reconnecting...
VERBOSE: Trying to connect to 192.168.16.2.
VERBOSE: Successfully connected to 192.168.16.2:4335 over IPv4.
netopeer2-server[3900]: Resolving unresolved data nodes and their constraints...
netopeer2-server[3900]: All data nodes and constraints resolved.
ERROR: Session 326: reading from the TLS session failed (SSL code 5).
ERROR: Failed to SSL_shutdown
VERBOSE: Trying to connect to 192.168.16.2.
VERBOSE: Successfully connected to 192.168.16.2:4335 over IPv4.
netopeer2-server[3900]: Resolving unresolved data nodes and their constraints...
netopeer2-server[3900]: All data nodes and constraints resolved.
ERROR: Call Home client "tls_ch_client" session 327 established.
ERROR: Session 327: communication socket unexpectedly closed (OpenSSL).
netopeer2-server[3900]: Session 327: thread 2 event session terminated.

....
this continues....

This is acceptable behaviour.
However, we observed the netopeer2-server has a number of opened sockets:

 2497 root     23712 S    /usr/sbin/netopeer2-server -v 1

root@box:/proc/2497/fd# for i in `seq 10`; do fds=`ls -1 | wc -l`; echo "FD count $fds"; sleep 2; done
FD count 667
FD count 669
FD count 669
FD count 670
FD count 672
FD count 672
FD count 673
FD count 675
FD count 675
FD count 676

root@box:/proc/2497/fd# ls -l | head -n 50
lrwx------    1 root     root            64 Mar 19 12:26 0 -> /dev/null
lrwx------    1 root     root            64 Mar 19 12:26 1 -> /dev/null
lrwx------    1 root     root            64 Mar 19 12:26 10 -> socket:[2826]
lrwx------    1 root     root            64 Mar 19 12:26 100 -> socket:[5035]
lrwx------    1 root     root            64 Mar 19 12:26 101 -> socket:[5057]
lrwx------    1 root     root            64 Mar 19 12:26 102 -> socket:[5061]
lrwx------    1 root     root            64 Mar 19 12:26 103 -> socket:[5062]
lrwx------    1 root     root            64 Mar 19 12:26 104 -> socket:[5073]
lrwx------    1 root     root            64 Mar 19 12:26 105 -> socket:[5074]
lrwx------    1 root     root            64 Mar 19 12:26 106 -> socket:[5075]
lrwx------    1 root     root            64 Mar 19 12:26 107 -> socket:[5089]
lrwx------    1 root     root            64 Mar 19 12:26 108 -> socket:[5090]
lrwx------    1 root     root            64 Mar 19 12:26 109 -> socket:[5091]
lrwx------    1 root     root            64 Mar 19 12:26 11 -> socket:[2829]
lrwx------    1 root     root            64 Mar 19 12:26 110 -> socket:[5102]
lrwx------    1 root     root            64 Mar 19 12:26 111 -> socket:[5103]
lrwx------    1 root     root            64 Mar 19 12:26 112 -> socket:[5123]
lrwx------    1 root     root            64 Mar 19 12:26 113 -> socket:[5129]
...
lrwx------    1 root     root            64 Mar 19 12:26 140 -> socket:[5359]
lrwx------    1 root     root            64 Mar 19 12:26 141 -> socket:[5369]
lrwx------    1 root     root            64 Mar 19 12:26 142 -> socket:[5372]

This count eventually grew upto 1024 & the system was frozen (no more fds could be opened).

Some analysis:
After above failure in SSL layer, the server goes to nc_session_free(struct nc_session *session, void (*data_free)(void *)). This function cleans the SSL context, however it doesn't closes the socket which is set in the session->ti.tls by SSL_set_fd(). Then the call-home thread kiks in and nc_sock_connect(const char* host, uint16_t port) create another socket.

I have created the pull request: #52. Still, there can be other reasons why this is done this way, so I'm open for changes.

Regards,
Darshana.

nc_connect_inout

Is anybody using this or can provide an example of how to use? Or tell me if I'm using it inappropriately. I am trying to establish a NETCONF session between a server and a client on the same machine.

In my client, when I replace a working SSH connect:

    s = nc_connect_ssh("localhost", 0, NULL);

with:

    int sock[2];
    socketpair(AF_UNIX, SOCK_STREAM, 0, sock);
    s = nc_connect_inout(sock[0], sock[1], NULL);

NULL is returned and libnetconf2 reports: Missing <session-id> in server's <hello>. Debug logging seems to show messages flowing in both directions, it just looks like the server reply fails validation. Any information is appreciated. Thanks,

-- Kevin

processing yang-library data instead of <hello> content

Client side code currently parses <hello> content to get information about the server's schemas and to create libyang context. This way all the schemas are implemented on the client side while they are just imported on the server side.

If the server supports ietf-yang-library the primary way to get info about the server's schemas should be to get the yang-library data and create context according to it. yang-library data include information if the schema is imported or implemented so the context on the client side will be really the same as on the server side.

Compile errors in session_client.c

From the devel branch v 0.9.9.

ly_verb is defined as
void ly_verb(LY_LOG_LEVEL level);

so the following two lines produce errors

libnetconf2/src/src/session_client.c: In function 'nc_ctx_fill_cpblts':
libnetconf2/src/src/session_client.c:481:18: error: void value not ignored as it ought to be
verb = ly_verb(LY_LLSILENT);
^
libnetconf2/src/src/session_client.c: In function 'nc_ctx_fill_yl':
libnetconf2/src/src/session_client.c:676:18: error: void value not ignored as it ought to be
verb = ly_verb(LY_LLSILENT);
^

OpenSSL 1.1 build issues

While trying to compile libnetconf2 using OpenSSL 1.1:

Downloads/libnetconf2-master/src/session.c: In function ‘nc_thread_destroy’:
Downloads/libnetconf2-master/src/session.c:1210:5: warning: ‘ERR_remove_thread_state’ is deprecated (declared at /usr/local/include/openssl/err.h:246) [-Wdeprecated-declarations]
Downloads/libnetconf2-master/src/session.c: At top level:
Downloads/libnetconf2-master/src/session.c:1049:1: warning: ‘tls_dyn_create_func’ defined but not used [-Wunused-function]
Downloads/libnetconf2-master/src/session.c:1064:1: warning: ‘tls_dyn_lock_func’ defined but not used [-Wunused-function]
Downloads/libnetconf2-master/src/session.c:1076:1: warning: ‘tls_dyn_destroy_func’ defined but not used [-Wunused-function]
[ 33%] Building C object CMakeFiles/netconf2.dir/src/session_client.c.o
[ 38%] Building C object CMakeFiles/netconf2.dir/src/session_server.c.o
[ 44%] Building C object CMakeFiles/netconf2.dir/src/time.c.o
[ 50%] Building C object CMakeFiles/netconf2.dir/src/session_client_ssh.c.o
[ 55%] Building C object CMakeFiles/netconf2.dir/src/session_server_ssh.c.o
[ 61%] Building C object CMakeFiles/netconf2.dir/src/session_client_tls.c.o
Downloads/libnetconf2-master/src/session_client_tls.c: In function ‘tlsauth_verify_callback’:
Downloads/libnetconf2-master/src/session_client_tls.c:40:20: error: storage size of ‘store_ctx’ isn’t known
Downloads/libnetconf2-master/src/session_client_tls.c:41:17: error: storage size of ‘obj’ isn’t known
Downloads/libnetconf2-master/src/session_client_tls.c:74:9: warning: ‘X509_CRL_get_nextUpdate’ is deprecated (declared at /usr/local/include/openssl/x509.h:709) [-Wdeprecated-declarations]
Downloads/libnetconf2-master/src/session_client_tls.c:80:13: warning: implicit declaration of function ‘X509_OBJECT_free_contents’ [-Wimplicit-function-declaration]
Downloads/libnetconf2-master/src/session_client_tls.c:116:41: error: dereferencing pointer to incomplete type
Downloads/libnetconf2-master/src/session_client_tls.c:41:17: warning: unused variable ‘obj’ [-Wunused-variable]
Downloads/libnetconf2-master/src/session_client_tls.c:40:20: warning: unused variable ‘store_ctx’ [-Wunused-variable]
Downloads/libnetconf2-master/src/session_client_tls.c: In function ‘nc_client_tls_update_opts’:
Downloads/libnetconf2-master/src/session_client_tls.c:396:9: warning: ‘TLSv1_2_client_method’ is deprecated (declared at /usr/local/include/openssl/ssl.h:1610) [-Wdeprecated-declarations]
Downloads/libnetconf2-master/src/session_client_tls.c:434:24: error: dereferencing pointer to incomplete type
make[2]: *** [CMakeFiles/netconf2.dir/src/session_client_tls.c.o] Error 1
make[1]: *** [CMakeFiles/netconf2.dir/all] Error 2
make: *** [all] Error 2
[ 5%] Building C object CMakeFiles/netconf2.dir/src/session_client_tls.c.o
Downloads/libnetconf2-master/src/session_client_tls.c: In function ‘tlsauth_verify_callback’:
Downloads/libnetconf2-master/src/session_client_tls.c:40:20: error: storage size of ‘store_ctx’ isn’t known
Downloads/libnetconf2-master/src/session_client_tls.c:41:17: error: storage size of ‘obj’ isn’t known
Downloads/libnetconf2-master/src/session_client_tls.c:74:9: warning: ‘X509_CRL_get_nextUpdate’ is deprecated (declared at /usr/local/include/openssl/x509.h:709) [-Wdeprecated-declarations]
Downloads/libnetconf2-master/src/session_client_tls.c:80:13: warning: implicit declaration of function ‘X509_OBJECT_free_contents’ [-Wimplicit-function-declaration]
Downloads/libnetconf2-master/src/session_client_tls.c:116:41: error: dereferencing pointer to incomplete type
Downloads/libnetconf2-master/src/session_client_tls.c:41:17: warning: unused variable ‘obj’ [-Wunused-variable]
Downloads/libnetconf2-master/src/session_client_tls.c:40:20: warning: unused variable ‘store_ctx’ [-Wunused-variable]
Downloads/libnetconf2-master/src/session_client_tls.c: In function ‘nc_client_tls_update_opts’:
Downloads/libnetconf2-master/src/session_client_tls.c:396:9: warning: ‘TLSv1_2_client_method’ is deprecated (declared at /usr/local/include/openssl/ssl.h:1610) [-Wdeprecated-declarations]
Downloads/libnetconf2-master/src/session_client_tls.c:434:24: error: dereferencing pointer to incomplete type
make[2]: *** [CMakeFiles/netconf2.dir/src/session_client_tls.c.o] Error 1
make[1]: *** [CMakeFiles/netconf2.dir/all] Error 2
make: *** [all] Error 2

Not able to run editconfig of Python library

I am trying to run " libnetconf2/python/examples/editconfig.py" but getting below issue,please help me in this.I am able to run get & serverinfo python examples:

~/libnetconf2/python/examples$ ./editconfig1.py
hostname: localhost
port : 830
username: vagrant
vagrant@localhost password :
libyang[1]: Previous warning generated by XPath subexpression[3] "map-type = 'ietf-x509-cert-to-name:specified'".
Traceback (most recent call last):
File "./editconfig1.py", line 66, in
print(data.print_mem(ly.LYD_XML, ly.LYP_FORMAT | ly.LYP_WITHSIBLINGS))
NameError: name 'data' is not defined

Missing libssh_threads path in dependency check

If the dependent library paths are different than default, SSH_LIBRARY library path should be included before loading -lssh_threads library.

File : CMakeLists.txt

dependencies - libssh
if(ENABLE_SSH)
find_package(LibSSH 0.6.4 REQUIRED)
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DNC_ENABLED_SSH ${LIBSSH_DEFINITIONS}")
target_link_libraries(netconf2 ${LIBSSH_LIBRARIES} -lssh_threads -lcrypt)
include_directories(${LIBSSH_INCLUDE_DIRS})
endif()

dependencies - libssh
if(ENABLE_SSH)
find_package(LibSSH 0.6.4 REQUIRED)
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DNC_ENABLED_SSH ${LIBSSH_DEFINITIONS}")
target_link_libraries(netconf2 ${LIBSSH_LIBRARIES} -L${SSH_LIBRARY} -lssh_threads -lcrypt)
include_directories(${LIBSSH_INCLUDE_DIRS})
endif()

add mechanism for updating client schema context

It is possible for server to modify its supported schemas (YANG 1.1 RFC) and the client may also want to support the new context. libnetconf2 does not allow this currently.

The implementation can add one new function (*_update), which would compare its current module-set-id with the one received from the server and if they would not match, try to update the context. If an application required it already, it can implement this completely outside libnetconf2 (it can access the context). Nevertheless, libnetconf2 already implements several ways of retrieving schemas, at least some of which the application would duplicate.

server SSH hangs

I tried to write the main function

ly_ctx = ly_ctx_new(TESTS_DIR"/../schemas");
nc_server_init(ly_ctx);
snode = ly_ctx_get_node(ly_ctx, NULL, "/ietf-netconf:get");
nc_set_rpc_callback(snode, op_get);
ly_ctx_get_module(....) and start ssh server.

I use the MG-SOFT client to connect ssh server.
client send rpc package:

<rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" essage-id="2">
  <get>
    <filter>
      <yanglib:modules-state xmlns:yanglib="urn:ietf:params:xml:ns:yang:ietf-yang-library"/>
    </filter>
  </get>
</rpc>

in op_get function:

lyd_print_file(stdout, root, LYD_XML, LYP_WITHSIBLINGS);

print:

<get xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
<data>
<modules-state xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-library">
<module>
<name>ietf-yang-metadata</name>
<revision>2016-08-05</revision>
<namespace>urn:ietf:params:xml:ns:yang:ietf-yang-metadata</namespace>
<conformance-type>import</conformance-type>
</module>

<module>
<name>yang</name>
<revision>2017-02-20</revision>
<namespace>urn:ietf:params:xml:ns:yang:1</namespace>
<conformance-type>implement</conformance-type>
</module>

<module>
<name>ietf-inet-types</name>
<revision>2013-07-15</revision>
<namespace>urn:ietf:params:xml:ns:yang:ietf-inet-types</namespace>
<conformance-type>import</conformance-type>
</module>

<module>
<name>ietf-yang-types</name>
<revision>2013-07-15</revision>
<namespace>urn:ietf:params:xml:ns:yang:ietf-yang-types</namespace>
<conformance-type>import</conformance-type>
</module>

<module>
<name>ietf-yang-library</name>
<revision>2016-06-21</revision>
<namespace>urn:ietf:params:xml:ns:yang:ietf-yang-library</namespace>
<conformance-type>implement</conformance-type>
</module>

<module>
<name>ietf-netconf</name>
<revision>2011-06-01</revision>
<namespace>urn:ietf:params:xml:ns:netconf:base:1.0</namespace>
<feature>writable-running</feature>
<feature>candidate</feature>
<feature>rollback-on-error</feature>
<feature>validate</feature>
<feature>startup</feature>
<feature>xpath</feature>
<conformance-type>implement</conformance-type>
</module>

<module>
<name>ietf-netconf-monitoring</name>
<revision>2010-10-04</revision>
<namespace>urn:ietf:params:xml:ns:yang:ietf-netconf-monitoring</namespace>
<conformance-type>implement</conformance-type>
</module>

<module>
<name>ietf-netconf-with-defaults</name>
<revision>2011-06-01</revision>
<namespace>urn:ietf:params:xml:ns:yang:ietf-netconf-with-defaults</namespace>
<conformance-type>implement</conformance-type>
</module>

<module>
<name>notifications</name>
<revision>2008-07-14</revision>
<namespace>urn:ietf:params:xml:ns:netconf:notification:1.0</namespace>
<conformance-type>implement</conformance-type>
</module>

<module>
<name>nc-notifications</name>
<revision>2008-07-14</revision>
<namespace>urn:ietf:params:xml:ns:netmod:notification</namespace>
<conformance-type>implement</conformance-type>
</module>

<module>
<name>ietf-netconf-notifications</name>
<revision>2012-02-06</revision>
<namespace>urn:ietf:params:xml:ns:yang:ietf-netconf-notifications</namespace>
<conformance-type>implement</conformance-type>
</module>

<module>
<name>if</name>
<revision>2017-06-21</revision>
<schema>file:///opt/libnetconf2/schemas/if.yang</schema>
<namespace>utstar:tn705e:if</namespace>
<conformance-type>implement</conformance-type>
</module>

<module-set-id>13</module-set-id>
</modules-state>
</data>
</get>

after I call nc_server_reply_data(root, nc_wd, NC_PARAMTYPE_FREE) server SSH hangs.

I do not know what's wrong. can you help me? thank you!

Invalid RPC tags cause netopeer2-server to SIGSEGV.

Hello,

If you send an invalid RPC such as the one below, the latest libnetconf2 crashes.

<?xml version="1.0" encoding="utf-8"?>
<rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="5">
  <blah-blah/>
</rpc>

The error seems to originate in src/messages_server.c:470 where it attempts to get the error message path. Call to ly_errpath can return NULL which isn't verified in the switch case for LYVE_INELEM which crashes with SIGSEGV.

For now, I've patched those lines with:

        case LYVE_INELEM:
            str = ly_errmsg(ctx);
            if (str) {
                if (strstr(str, "Unknown")) {
                    e = nc_err(NC_ERR_UNKNOWN_ELEM, NC_ERR_TYPE_PROT, ly_errpath(ctx));
                } else if (strstr(str, "Unsupported")) {
                    e = nc_err(NC_ERR_OP_NOT_SUPPORTED, NC_ERR_TYPE_APP);
                    return e;
                }
            } else {
                e = nc_err(NC_ERR_UNKNOWN_ELEM, NC_ERR_TYPE_PROT, ly_errpath(ctx));
            }
            break;

I could be wrong. Could you please have a look at it instead?

Thanks,
Anandu

The rpc-reply element tag is not always in the correct namespace

Summary:
If the client uses a prefix for the NETCONF namespace then the rpc-reply element tag is not in the NETCONF namespace

Details:
The Python library ncclient sends an rpc beginning as follows. Note that there is an 'nc' prefix on the rpc tag:
<nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:b9e28b4b-6bf2-4606-9e5a-9bc08074d609"><nc:edit-config>

The reply from libnetconf2 is as follows:
<rpc-reply xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:b9e28b4b-6bf2-4606-9e5a-9bc08074d609"><ok/></rpc-reply>

Note that there is no 'nc' prefix on the rpc-reply tag. The response is not recognized and ncclient times out. I believe that it is because the rpc-reply element tag is not in the NETCONF namespace.

I think that the correct response would need to have a prefix on the rpc-reply tag:
<nc:rpc-reply xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0"

NETCONF seems to require that the rpc attributes are echoed back verbatim.

I made an attempt to fix it, however I don't suppose my change is very clean. I have done limited testing so far, but I was able to get further with ncclient.

Thanks
fix_reply_namespace .patch.txt

Use CLOCK_MONOTONIC at some places instead of CLOCK_REALTIME

As per my understanding, at many places monotonic clock should be used instead of real time clock.

Like in the main loop of nc_ps_poll. Timeouts are affected using real time clock when changing TOD of the system.

Also, last_rpc & idle timeout calculations for call home connections should also be using monotonic clocks.

I'm saying this because I was using a very slightly modified version of nc_ps_poll in my code. On changing the system time backwards (say by 2 hours), thread having nc_ps_lock got stuck in a loop for 2 hours, and other threads were deprived of the lock during that time.
Changing to monotonic clock in nc_ps_poll solved the issue.

I thought to change nc_gettimespec to use monotonic clock, but it's being used for pthread_mutex_timedlock & pthread_cond_timedwait which need absolute time.

So selective changes need to be made.

Failed to add session to poll context

I have a netconf server which listens on port 830, and accept client connection and process RPC.
For first session nc_ps_add_session() is successfull and session is added to the poll context, but nc_ps_add_session() is failing always with timeout error for second session.

SESSION 1 :

New incomming session -->
Received Hello message from client. Adding session to poll context.
Successfully added a new session to poll context.

SESSION 2:
New incomming session
Received Hello message from client. Adding session to poll context.
ERROR: nc_ps_add_session: failed to wait for a pollsession condition (Connection timed out).
Failed to add new session to poll context.Retrying..
ERROR: nc_ps_add_session: failed to wait for a pollsession condition (Connection timed out).
Failed to add new session to poll context.Retrying..
ERROR: nc_ps_add_session: failed to wait for a pollsession condition (Connection timed out).
Failed to add new session to poll context.Retrying..
ERROR: nc_ps_add_session: failed to wait for a pollsession condition (Connection timed out).
Failed to add new session to poll context.Retrying..
ERROR: nc_ps_add_session: failed to wait for a pollsession condition (Connection timed out).
Failed to add new session to poll context.Retrying..
Failed to add new session to poll context. Clossing SSh session.

EDIT:
If the issue is fixed, please point the commit ID .

Issue parsing module

Hello,

I am facing a couple of issues in parsing a module to connect to a NETCONF server using netopeergui.

The capability is "urn:yumaworks:params:xml:ns:netconf:config-id?id=401", which according to my understanding is allowed per RFC 6020.

However whenever I try to connect I receive the following error: "lib ERROR: Unknown capability "urn:yumaworks:params:xml:ns:netconf:config-id?id=401" could not be parsed.

After some basic tracing methods I found out the this error occurs in the session_client.c script, which dictates that the capability has to have "module= " in its body, which is not the case. This consequently interrupts the whole connection process.

Is there a way that I can edit/overwrite that without causing other issues that might occur accordingly?

Thanks,

Maged

An idea about 'nc_rpc_clb'

Hi,
I have met a question about handling RPC request. I want to use one common callback for all my RPC requests (in my case, just dispatch all RPC request to sysrepo with sr_rpc_send). But now I must set the callback function pointer to lys_node for all RPC, so I think we should allow users to give a global callback for handling RPC request, and if this global callback is not set, we can get callback function from lys_node to make sure current API still works.
If you agree with this idea, I can work on it and push the code to you.

missing CPackConfig.cmake file?

CMake Error at CMakeLists.txt:36 (include):
include could not find load file:

CPackConfig.cmake

am I just missing a step that creates this file or is it missing?

Correspondence of NETCONF Capability

hi,I just learned libnetconf2 library. so I do not know how to make it support:
writable-running
:candidate
:confirmed-commit:1.1

can you help me?thanks!

netopeer2-server segfaults on `get` RPC

Hi, it seems that the recent commit which fixes handlig of decimal64 list keys actually works. Here's how a get from the CLI ends up:

Thread 6 "netopeer2-serve" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb4afeb40 (LWP 1078)]
0xb7668b60 in resolve_partial_json_data_nodeid (nodeid=0xb6514c20 "/czechlight-roadm-v2:roadm/configuration/channel[center-frequency='195.3']/center-frequency", 
    llist_value=0xb4afe05c "195.3", '0' <repeats 12 times>, "1136868377216160297393798828125", '0' <repeats 152 times>..., start=0x30303030, options=1, 
    parsed=0xb4afdf9c) at /home/jkt/work/prog/buildroot/out-epia-geode/build/libyang-custom/src/resolve.c:2230
2230    /home/jkt/work/prog/buildroot/out-epia-geode/build/libyang-custom/src/resolve.c: No such file or directory.
(gdb) bt
#0  0xb7668b60 in resolve_partial_json_data_nodeid (nodeid=0xb6514c20 "/czechlight-roadm-v2:roadm/configuration/channel[center-frequency='195.3']/center-frequency", 
    llist_value=0xb4afe05c "195.3", '0' <repeats 12 times>, "1136868377216160297393798828125", '0' <repeats 152 times>..., start=0x30303030, options=1, 
    parsed=0xb4afdf9c) at /home/jkt/work/prog/buildroot/out-epia-geode/build/libyang-custom/src/resolve.c:2230
#1  0xb76d234e in lyd_new_path (data_tree=0x30303030, ctx=0x88c0420, 
    path=0xb6514c20 "/czechlight-roadm-v2:roadm/configuration/channel[center-frequency='195.3']/center-frequency", value=0xb4afe05c, 
    value_type=LYD_ANYDATA_CONSTSTRING, options=1) at /home/jkt/work/prog/buildroot/out-epia-geode/build/libyang-custom/src/tree_data.c:1229
#2  0x08057951 in opget_build_subtree_from_sysrepo (ds=0x892d320, root=0xb4afe110, subtree_xpath=0xb6503520 "/czechlight-roadm-v2:roadm")
    at /home/jkt/work/prog/buildroot/out-epia-geode/build/netopeer2-server-custom/server/op_get_config.c:62
#3  0x30303030 in ?? ()
#4  0x30303030 in ?? ()
#5  0x30303030 in ?? ()
#6  0x30303030 in ?? ()
#7  0x30303030 in ?? ()
#8  0x30303030 in ?? ()
#9  0x30303030 in ?? ()
#10 0x30303030 in ?? ()
#11 0x30303030 in ?? ()
#12 0x30303030 in ?? ()
#13 0x30303030 in ?? ()
#14 0x30303030 in ?? ()
#15 0x30303030 in ?? ()
#16 0x30303030 in ?? ()
#17 0x30303030 in ?? ()
#18 0x30303030 in ?? ()
#19 0x30303030 in ?? ()
#20 0x30303030 in ?? ()
#21 0x30303030 in ?? ()
#22 0x30303030 in ?? ()
#23 0x30303030 in ?? ()
#24 0x30303030 in ?? ()
#25 0x30303030 in ?? ()
#26 0x30303030 in ?? ()
#27 0x30303030 in ?? ()
#28 0x00000000 in ?? ()

A major part of the stack trace looks like garbage.

This is with freshly rebuild libyang/sysrepo/libnetconf2/netopeer2 from devel branches.

netconf packet head

Hi ,
we find all netconf packet without the following data:

and some netconf server can not parse them.

can you add the data before all neconf packet.

RFC 6241's example describe as :
The following example shows a capability exchange. Data sent by the
NETCONF client are marked with "C:", and data sent by the NETCONF
server are marked with "S:".
S:
S:
S:
S:
S: urn:ietf:params:netconf:base:1.1
S:
S:
S: urn:ietf:params:ns:netconf:capability:startup:1.0
S:
S:
S: 4
S:
S: ]]>]]>
C:
C:
C:
C:
C: urn:ietf:params:netconf:base:1.1
C:
C:
C:
C: ]]>]]>

Thanks~

USERAUTH BANNER on login

I was trying to get USERAUTH BANNER on login.

The code has an API named nc_server_ssh_endpt_set_banner which sets SSH banner for the Netconf server to send to a client when connecting. The API doesn't seems to work in my application.
Attached are client-side logs with debug logging. I set the banner string as This is a banner!!

Also, as I searched over the internet for underlying library (libssh) used here, it does not have support for the warning/MOTD/issue banner. This is based on the absence of any code for sending of the SSH_MSG_USERAUTH_BANNER / SSH2_MSG_USERAUTH_BANNER message from the server.
Libssh seems to use “banner” to mean protocol version (i.e. the string initially exchanged by client and server to determine compatibility SSHv1/SSHv2). Though the same has been supported in libssh2.

If this feature not covered in libnetconf2, is there any plan to support the same in near future or upgrade the libssh library to libssh2 to make such small feature work or any workaround that I can use here.
Client Logs.txt

ssh callhome occasionally does not exit session

Occasionally on low powered (?) hardware, when nc_session_free signals ch_lock, the timedwait does not exit in a timely manner and nc_session_free completes thus destroying the mutex and condition variable. The pthread_cond_timedwait call in nc_server_ch_client_thread_session_cond_wait never exits and the callhome retry never happens as the session is effectively not closed.

pthread_cond_timedwait behavior when the underlying condition variable is destroyed is undefined.

linux used is 4.1.38 on aarch64 with 2 cores.
libnetconf2 is current to nov 2017. No further commits were found that were relevant to this issue.

This was exhibited with ncclient (master, 0.5.3 is inadequate for callhome) and netopeer2 server where on client program shutdown the session was closed without stopping the tcp listener first.

A possible tested solution is attached.

A better solution may include another cond var/mutex for signalling back to nc_session_free that the session is no longer in use and destruction can occur.

ssh_callhome_race.patch.gz

segmentation fault on closure of call home connection

we are seeing a crash when the call home connection is closed from the other end, the ch_lock and ch_cond parameters are NULL and nc_session_free tries to grab a lock on NULL ch_lock. I am pasting the stack trace below. from the code analysis it looks like the locks are not being allocated for the child sessions on the parent call home session. Please advise.

#0 __GI___pthread_mutex_lock (mutex=0x0) at ../nptl/pthread_mutex_lock.c:65
#1 0x00007ffff76570d6 in nc_session_free (session=0x7fffe4001250, data_free=0x407acf <free_ds>)
at session.c:622
#2 0x0000000000408e7c in np2srv_del_session_clb (session=0x7fffe4001250) at main.c:756
#3 0x000000000040a5fd in worker_thread (arg=0x6e0240) at main.c:1302
#4 0x000000000040b056 in main (argc=3, argv=0x7fffffffe178) at main.c:1541

Cannot connect OpenDaylight to netopeer2 with keyboard-interactive authentication

libnetconf2 seems to handle keyboard-interactive authentication incorrectly, as it responds with a spurious SSH_MSG_USERAUTH_FAILURE when asked for keyboard-interactive authentication.
This results in handshake state error, with the an extra SSH_MSG_USERAUTH_FAILURE being delivered to an authenticated session. While OpenSSH reacts with SSH_MSG_UNIMPLEMENTED, Apache MINA SSHD throws an IllegalStateException and terminates the session.

memory leak in nc_ps_poll

In nc_ps_poll, line 1541, the rpc object result from nc_server_recv_rpc_io maybe leaked.

    if (ret == NC_PSPOLL_RPC) {
        ret = nc_server_recv_rpc_io(cur_session, timeout, &rpc); // not free the rpc object
        if (ret & (NC_PSPOLL_ERROR | NC_PSPOLL_BAD_RPC)) {
            if (cur_session->status != NC_STATUS_RUNNING) {
                ret |= NC_PSPOLL_SESSION_TERM | NC_PSPOLL_SESSION_ERROR;
                cur_ps_session->state = NC_PS_STATE_INVALID;
            } else {
                cur_ps_session->state = NC_PS_STATE_NONE;
            }
        } else {

Error in building libnetconf2

Hello,

I am working on installing libnetconf2 on a docker container to effectively install the new Netopeer2GUI.
According to the install instructions, the devel branches have to be used in the time being.

However, whenever I try to install libnetconf2 (after installing libyang) I get the following error:

/libnetconf2/python/rpc.c:22:31: fatal error: libyang/swigpyrun.h: No such file or directory
 #include <libyang/swigpyrun.h>
                               ^
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
make[2]: *** [python/CMakeFiles/pyapi] Error 1
make[1]: *** [python/CMakeFiles/pyapi.dir/all] Error 2
make: *** [all] Error 2

I have attached the Dockerfile container for your reference (remove the '.txt' extension to run in linux).

Dockerfile.txt

You can just run the following command inside the folder with the Dockerfile:

docker pull ubuntu:trusty
docker build -t ubuntu:trusty .

Let me know if you need any further information from my end, or if you would want to raise the issue in libyang

Thank you,

Maged

need to check null pointer

if (strcmp(cpblt->name, "capability") && cpblt->ns && cpblt->ns->value &&

cpblt->name may be null pointer when receive illegal packet. Similar issues maybe exist some where else.

illegal packet example, missing \r\n in first line.

<?xml version="1.0\" encoding="UTF-8"?>
        <hello xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">\r\n
        <capabilities>\r\n
        <capability>urn:ietf:params:netconf:base:1.1</capability>\r\n
        <capability>urn:ietf:params:netconf:base:1.0</capability>\r\n
        </capabilities>\r\n
        </hello>]]>]]>\r\n

Compilation error on MAC OSX

I encountered errors when compiling libnetconf2 on my MAC, it seems that the MAC OSX does not support pthread_rwlock_t and CLOCK_REALTIME. The error log is :

sunlinhuideAir:build sunlinhui$ make
Scanning dependencies of target netconf2
[ 4%] Building C object CMakeFiles/netconf2.dir/src/io.c.o
In file included from /Users/sunlinhui/Documents/workspace/libnetconf2/src/io.c:29:
In file included from /Users/sunlinhui/Documents/workspace/libnetconf2/src/libnetconf.h:22:
/Users/sunlinhui/Documents/workspace/libnetconf2/src/session_p.h:195:5: error:
unknown type name 'pthread_spinlock_t'; did you mean 'pthread_rwlock_t'?
pthread_spinlock_t sid_lock;
^~~~~~~~~~~~~~~~~~
pthread_rwlock_t
/usr/include/sys/_pthread/_pthread_rwlock_t.h:30:35: note: 'pthread_rwlock_t'
declared here
typedef __darwin_pthread_rwlock_t pthread_rwlock_t;
^
/Users/sunlinhui/Documents/workspace/libnetconf2/src/io.c:459:19: warning:
implicit declaration of function 'ppoll' is invalid in C99
[-Wimplicit-function-declaration]
ret = ppoll(&fds, 1, (timeout == -1 ? NULL : &ts_timeout), &...
^
1 warning and 1 error generated.
make[2]: *** [CMakeFiles/netconf2.dir/src/io.c.o] Error 1
make[1]: *** [CMakeFiles/netconf2.dir/all] Error 2
make: *** [all] Error 2

Any idea?

Undefined behavior in va_start

Calling va_start(X) is undefined when the last argument has a type which is incompatible from the type that results from the default argument promotion:

[6/27] Building C object CMakeFiles/netconf2.dir/src/messages_server.c.o
../src/messages_server.c:146:18: warning: passing an object that undergoes default argument promotion to 'va_start' has undefined behavior [-Wvarargs]
    va_start(ap, tag);
                 ^
../src/messages_server.c:126:15: note: parameter of type 'NC_ERR' (aka 'enum NC_ERROR') is declared here
nc_err(NC_ERR tag, ...)
              ^
1 warning generated.
[9/27] Building C object CMakeFiles/netconf2.dir/src/io.c.o
../src/io.c:983:18: warning: passing an object that undergoes default argument promotion to 'va_start' has undefined behavior [-Wvarargs]
    va_start(ap, type);
                 ^
../src/io.c:961:54: note: parameter of type 'NC_MSG_TYPE' (aka 'enum NC_MSG_TYPE') is declared here
nc_write_msg(struct nc_session *session, NC_MSG_TYPE type, ...)

I think that a most elegant fix is specifying the NC_MSG_TYPE enum size to be int; however, that's an obvious ABI break because it changes sizeof(NC_MSG_TYPE).

Some workarounds are possible, including adding another tag argument just for va_args. However, this is impossible if a function is exported because that, too, is an ABI break.

callback to get unknown schema

When libnetconf is not able to get a schema used by the server (via get-schema or from a local directory) it should, instead of just printing error/warning, use a callback provided by the caller to get the schema. Caller can have another sources (including user interaction user) from where the schema can be obtained.

There shouldn't be any issue with timeouts because in this phase the NETCONF session is actually already established and just the client side context is being set.

Memory leak in nc_recv_client_hello/parse_cpblts

  1. connect to the netopeer2-server from a netopeer2-cli over SSH, no password
  2. immediately quit
    Versions from devel branches as of this Wednesday.
$ netopeer2-cli 
> connect --host ... --ssh --login root
Interactive SSH Authentication
Type your password:
Password: 
> quit

=================================================================
==18891==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 2712 byte(s) in 32 object(s) allocated from:
    #0 0x7fbaf544db78 in malloc (/usr/lib/gcc/x86_64-pc-linux-gnu/6.3.0/libasan.so.3+0xd2b78)
    #1 0x7fbaf3fe9aad in strndup (/lib64/libc.so.6+0x83aad)
    #2 0x7fbaf4c5bdf1 in parse_cpblts /home/jkt/work/cesnet/gerrit/github/CESNET/libnetconf2/src/session.c:984
    #3 0x7fbaf4c5c7de in nc_recv_client_hello /home/jkt/work/cesnet/gerrit/github/CESNET/libnetconf2/src/session.c:1095
    #4 0x7fbaf4c5cf96 in nc_handshake /home/jkt/work/cesnet/gerrit/github/CESNET/libnetconf2/src/session.c:1201
    #5 0x7fbaf4c80e51 in nc_connect_ssh /home/jkt/work/cesnet/gerrit/github/CESNET/libnetconf2/src/session_client_ssh.c:1526
    #6 0x40d204 in cmd_connect_listen_ssh /home/jkt/work/cesnet/gerrit/github/CESNET/Netopeer2/cli/commands.c:1393
    #7 0x4130cb in cmd_connect_listen /home/jkt/work/cesnet/gerrit/github/CESNET/Netopeer2/cli/commands.c:2444
    #8 0x4132d4 in cmd_connect /home/jkt/work/cesnet/gerrit/github/CESNET/Netopeer2/cli/commands.c:2476
    #9 0x406728 in main /home/jkt/work/cesnet/gerrit/github/CESNET/Netopeer2/cli/main.c:191
    #10 0x7fbaf3f868b3 in __libc_start_main (/lib64/libc.so.6+0x208b3)
    #11 0x405928 in _start (/opt/nc/bin/netopeer2-cli+0x405928)

server: segfault when client with ssh multisession crashes

When a client that has opened multiple sessions that share the same SSH connection exits without closing the sessions, the server crashes.

Program terminated with signal SIGSEGV, Segmentation fault.
in nc_session_free (libnetconf2/src/session.c:713).
712                 if (session->flags & NC_SESSION_SSH_MSG_CB) {
713 >>                  for (siter = session->ti.libssh.next; siter->status != NC_STATUS_RUNNING; siter = siter->ti.libssh.next) {
714                         if (siter->ti.libssh.next == session) {
715                             ERRINT;
716                             break;
717                         }

It looks like there are 2 threads handling both sessions and they are calling nc_session_free at the same time. I tried to add locks to prevent an obvious race condition:

diff --git a/src/session.c b/src/session.c
index 0f3b253bdd25..fefa71f8664c 100644
--- a/src/session.c
+++ b/src/session.c
@@ -552,6 +552,13 @@ nc_session_free(struct nc_session *session, void (*data_free)(void *))
         } /* else failed to lock it, too bad */
     }
 
+#ifdef NC_ENABLED_SSH
+    /* avoid concurrent free by multiple threads of sessions that share the
+     * same SSH connection. */
+    if (session->ti_type == NC_TI_LIBSSH)
+        pthread_mutex_lock(session->io_lock);
+#endif
+
     if ((session->side == NC_CLIENT) && (session->status == NC_STATUS_RUNNING)) {
         /* cleanup message queues */
         /* notifications */
@@ -750,6 +757,11 @@ nc_session_free(struct nc_session *session, void (*data_free)(void *))
     lydict_remove(session->ctx, session->username);
     lydict_remove(session->ctx, session->host);
 
+#ifdef NC_ENABLED_SSH
+    if (session->ti_type == NC_TI_LIBSSH)
+        pthread_mutex_unlock(session->io_lock);
+#endif
+
     /* final cleanup */
     if ((session->side == NC_SERVER) && session->opts.server.rpc_lock) {
         if (rpc_locked) {

But another problem is still here:

netopeer2-server[2030]: Session 4: communication socket unexpectedly closed.
netopeer2-server[2030]: Session 4: failed to write reply.

The server exits after printing that message...

Maybe I missed something.

oper data callback option

Is there a way to get a callback for oper data using libnetconf2/netopeer2, so far I only see a way to get a callback from sysrepo when a data store config change occurred. If this currently does not exist - will it eventually be available and if so what is the time frame for it and what mechanism will be available to get a call back?

libnetconf2

I am unable to compile libnetconf2. The error is as below.

Linking C executable cmTC_a4e41
/usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_a4e41.dir/link.txt --verbose=1
/usr/bin/cc   -Wall -Wextra -fvisibility=hidden     CMakeFiles/cmTC_a4e41.dir/CheckSymbolExists.c.o  -o cmTC_a4e41 -rdynamic
CMakeFiles/cmTC_a4e41.dir/CheckSymbolExists.c.o: In function `main':
CheckSymbolExists.c:(.text+0x16): undefined reference to `pthread_create'
collect2: error: ld returned 1 exit status
CMakeFiles/cmTC_a4e41.dir/build.make:97: recipe for target 'cmTC_a4e41' failed
gmake[1]: Leaving directory '/home/jayshankar/Downloads/libnetconf2-master/build/CMakeFiles/CMakeTmp'
gmake[1]: *** [cmTC_a4e41] Error 1
Makefile:126: recipe for target 'cmTC_a4e41/fast' failed
gmake: *** [cmTC_a4e41/fast] Error 2

Determining if the function pthread_create exists in the pthreads failed with the following output:
Change Dir: /home/jayshankar/Downloads/libnetconf2-master/build/CMakeFiles/CMakeTmp

Run Build Command:"/usr/bin/gmake" "cmTC_4cb6f/fast"
/usr/bin/gmake -f CMakeFiles/cmTC_4cb6f.dir/build.make CMakeFiles/cmTC_4cb6f.dir/build
gmake[1]: Entering directory '/home/jayshankar/Downloads/libnetconf2-master/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_4cb6f.dir/CheckFunctionExists.c.o
/usr/bin/cc -Wall -Wextra -fvisibility=hidden -DCHECK_FUNCTION_EXISTS=pthread_create -o CMakeFiles/cmTC_4cb6f.dir/CheckFunctionExists.c.o -c /usr/share/cmake/Modules/CheckFunctionExists.c
Linking C executable cmTC_4cb6f
/usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_4cb6f.dir/link.txt --verbose=1
/usr/bin/cc -Wall -Wextra -fvisibility=hidden -DCHECK_FUNCTION_EXISTS=pthread_create CMakeFiles/cmTC_4cb6f.dir/CheckFunctionExists.c.o -o cmTC_4cb6f -rdynamic -lpthreads
/usr/bin/ld: cannot find -lpthreads
collect2: error: ld returned 1 exit status
CMakeFiles/cmTC_4cb6f.dir/build.make:97: recipe for target 'cmTC_4cb6f' failed
gmake[1]: Leaving directory '/home/jayshankar/Downloads/libnetconf2-master/build/CMakeFiles/CMakeTmp'
gmake[1]: *** [cmTC_4cb6f] Error 1
Makefile:126: recipe for target 'cmTC_4cb6f/fast' failed
gmake: *** [cmTC_4cb6f/fast] Error 2.

Issue in pthread_mutex_timedlock implementation

In src/session.c:

#ifndef HAVE_PTHREAD_MUTEX_TIMEDLOCK
int
pthread_mutex_timedlock(pthread_mutex_t *mutex, const struct timespec *abstime)
{
    int32_t diff;
    int rc;
    struct timespec cur, dur;

    /* Try to acquire the lock and, if we fail, sleep for 5ms. */
    while ((rc = pthread_mutex_trylock(mutex)) == EBUSY) {
        nc_gettimespec_real(&cur);

        if ((diff = nc_difftimespec(&cur, abstime)) < 1) {
            /* timeout */
            break;
        } else if (diff < 5) {
            /* sleep until timeout */
            dur = *abstime;
        } else {
            /* sleep 5 ms */
            dur.tv_sec = 0;
            dur.tv_nsec = 5000000;
        }

        nanosleep(&dur, NULL);
    }

    return rc;
}
#endif

abstime passed is time since epoch + some timeout value, i.e.,

gdb$ p *abstime
$8 = {
  tv_sec = 1505354115, 
  tv_nsec = 812837304
}

Now if code goes in else case, we nanosleep for (0s+5000000ns). But if we go inside else if (diff < 5), we sleep for (1505354115s+812837304ns). So it goes in kind of infinite sleep in this else if case.

Shouldn't we be sleeping for dur = *abstime - curtime (ignore the type-casting)?

handshake does not work when multiple sessions are created for the same ssh connection

handshake does not work when multiple sessions are created for the same ssh connection, we are trying to create multiple netconf sessions on the same ssh connection. The first netconf session goes through fine, we can see the capabilities and the hello reply back from server. however, the second session does not send the hello reply message back to the client, the client keeps on waiting for the reply message which never happens. I am using 0.9-r1 release. Please advise. I am pasting the logs from netopeer server to confirm this. I am using the latest git clone of netopeer2 server.

netopeer2-server[9027]: Accepted a connection on 0.0.0.0:830.
[2017/10/13 07:19:42.992085, 1] ssh_server_connection_callback: SSH client banner: SSH-2.0-Ganymed_262
[2017/10/13 07:19:42.992542, 1] ssh_analyze_banner: Analyzing banner: SSH-2.0-Ganymed_262
[2017/10/13 07:19:44.418126, 2] ssh_packet_newkeys: Received SSH_MSG_NEWKEYS
netopeer2-server[9027]: Received an SSH message "request-service" of subtype "ssh-userauth".
netopeer2-server[9027]: Received an SSH message "request-auth" of subtype "none".
netopeer2-server[9027]: Received an SSH message "request-auth" of subtype "password".
netopeer2-server[9027]: User "root" authenticated.
netopeer2-server[9027]: Received an SSH message "request-channel-open" of subtype "session".
netopeer2-server[9027]: Received an SSH message "request-channel" of subtype "subsystem".
netopeer2-server[9027]: Session 4: sending message:
urn:ietf:params:netconf:base:1.0urn:ietf:params:netconf:base:1.1urn:ietf:params:netconf:capability:writable-running:1.0urn:ietf:params:netconf:capability:candidate:1.0urn:ietf:params:netconf:capability:rollback-on-error:1.0urn:ietf:params:netconf:capability:validate:1.1urn:ietf:params:netconf:capability:startup:1.0urn:ietf:params:netconf:capability:xpath:1.0urn:ietf:params:netconf:capability:with-defaults:1.0?basic-mode=explicit&also-supported=report-all,report-all-tagged,trim,expliciturn:ietf:params:netconf:capability:notification:1.0urn:ietf:params:netconf:capability:interleave:1.0
netopeer2-server[9027]: Session 4: sending message:
urn:ietf:params:xml:ns:yang:ietf-yang-metadata?module=ietf-yang-metadata&revision=2016-08-05urn:ietf:params:xml:ns:yang:1?module=yang&revision=2017-02-20urn:ietf:params:xml:ns:yang:ietf-inet-types?module=ietf-inet-types&revision=2013-07-15urn:ietf:params:xml:ns:yang:ietf-yang-types?module=ietf-yang-types&revision=2013-07-15urn:ietf:params:xml:ns:yang:ietf-yang-library?module=ietf-yang-library&revision=2016-06-21&module-set-id=25urn:ietf:params:xml:ns:yang:ietf-netconf-acm?module=ietf-netconf-acm&revision=2012-02-22urn:ietf:params:xml:ns:netconf:base:1.0?module=ietf-netconf&revision=2011-06-01&features=writable-running,candidate,rollback-on-error,validate,startup,xpathurn:ietf:params:xml:ns:yang:ietf-netconf-notifications?module=ietf-netconf-notifications&revision=2012-02-06
netopeer2-server[9027]: Session 4: sending message:
urn:ietf:params:xml:ns:netconf:notification:1.0?module=notifications&revision=2008-07-14urn:ietf:params:xml:ns:netmod:notification?module=nc-notifications&revision=2008-07-14http://example.net/turing-machine?module=turing-machine&amp;revision=2013-12-27urn:ietf:params:xml:ns:yang:ietf-interfaces?module=ietf-interfaces&revision=2014-05-08urn:ietf:params:xml:ns:yang:iana-if-type?module=iana-if-type&revision=2014-05-08urn:ietf:params:xml:ns:yang:ietf-ip?module=ietf-ip&revision=2014-06-16urn:ietf:params:xml:ns:yang:ietf-keystore?module=ietf-keystore&revision=2016-10-31urn:ietf:params:xml:ns:yang:ietf-x509-cert-to-name?module=ietf-x509-cert-to-name&revision=2014-12-10urn:ietf:params:xml:ns:yang:ietf-ssh-server?module=ietf-ssh-server&revision=2016-11-02
netopeer2-server[9027]: Session 4: sending message:
urn:ietf:params:xml:ns:yang:ietf-tls-server?module=ietf-tls-server&revision=2016-11-02urn:ietf:params:xml:ns:yang:ietf-netconf-server?module=ietf-netconf-server&revision=2016-11-02&features=listen,ssh-listen,tls-listen,call-home,ssh-call-home,tls-call-homeurn:ietf:params:xml:ns:yang:iana-crypt-hash?module=iana-crypt-hash&revision=2014-08-06urn:ietf:params:xml:ns:yang:ietf-system?module=ietf-system&revision=2014-08-06&features=authentication,local-usershttp://mavenir.com/ns/MainConfig/1.1?module=MainConfig&amp;revision=2016-08-08urn:ietf:params:xml:ns:yang:ietf-netconf-monitoring?module=ietf-netconf-monitoring&revision=2010-10-04urn:ietf:params:xml:ns:yang:ietf-netconf-with-defaults?module=ietf-netconf-with-defaults&revision=2011-06-014
netopeer2-server[9027]: Session 4: sending message:
]]>]]>
[2017/10/13 07:19:45.986075, 2] grow_window: growing window (channel 43:100) to 1280000 bytes
netopeer2-server[9027]: Session 4: received message:
urn:ietf:params:netconf:base:1.0


netopeer2-server[9027]: Sending session-start request.
netopeer2-server[9027]: session-start request sent, waiting for response.
netopeer2-server[9027]: session-start response received, processing.
netopeer2-server[9027]: Sending event-notification request.
netopeer2-server[9027]: event-notification request sent, waiting for response.
netopeer2-server[9027]: event-notification response received, processing.
netopeer2-server[9027]: Generated new event (netconf-session-start).
netopeer2-server[9027]: Received an SSH message "request-channel-open" of subtype "session".
netopeer2-server[9027]: Received an SSH message "request-channel" of subtype "subsystem".

macos can't find 'shadow.h'

Hi, I want to compile libnetconf2 on macos using following command.

cmake -DOPENSSL_ROOT_DIR=/usr/local/opt/openssl -DOPENSSL_LIBRARIES=/usr/local/opt/openssl/lib ..
make

CMake can be done with waring CMP0042.
But when I try to make it, shadow.h can't be find.

[  4%] Building C object CMakeFiles/netconf2.dir/src/session_server_ssh.c.o
/Users/dy/Source/CESNET/libnetconf2/src/session_server_ssh.c:23:10: fatal error: 'shadow.h' file not found
#include <shadow.h>
         ^
1 error generated.
make[2]: *** [CMakeFiles/netconf2.dir/src/session_server_ssh.c.o] Error 1
make[1]: *** [CMakeFiles/netconf2.dir/all] Error 2

subtree filtering on server

Hi there,

Looking through the code it is not obvious if subtree filtering is supported by a feature of the server itself, or if the filter anyxml is simply passed to the callback and the callback is supposed to manage the parsing of the filter and apply it.

Some examples or description would be great, if they are available.

dmc

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.