Does anyone know where the LIBRETS documentation has gone?

It used to be here, but not any more:
www.crt.realtors.org/projects/rets/librets/documentation/devguide/

According to the build docs at:
https://github.com/NationalAssociationOfRealtors/libRETS/blob/master/doc/build/unix.txt

After installing dependencies, you should be able to install by running these commands:

% ./configure 
% make
% sudo make install

However, in both the clone from the repo and the source packages downloadable from here: http://nationalassociationofrealtors.github.io/libRETS/downloads/ the configure script does not exist.

I have developed a tool to get datas from RETS. It can work on Dotnet 4.0 under Windows 7 and Win 10;

But I try to run it on Mono under linux; It always get a exception.

Linux: Centos 7 64 bit and Mac OSX:
Mono 4.2.1
LibRETS 1.6.1 x64

Exceptions:

System.TypeInitializationException: The type initializer for 'librets.RetsSession' threw an exception.
---> System.TypeInitializationException: The type initializer for 'librets.libretsPINVOKE' threw an exception.
---> System.TypeInitializationException: The type initializer for 'SWIGExceptionHelper' threw an exception.
---> System.DllNotFoundException: librets-pinvoke.dll
at (wrapper managed-to-native) librets.libretsPINVOKE+SWIGExceptionHelper:SWIGRegisterExceptionCallbacks_librets (librets.libretsPINVOKE/SWIGExceptionHelper/ExceptionDelegate,librets.libretsPINVOKE/SWIGExceptionHelper/ExceptionDelegate,librets.libretsPINVOKE/SWIGExceptionHelper/ExceptionDelegate,librets.libretsPINVOKE/SWIGExceptionHelper/ExceptionDelegate,librets.libretsPINVOKE/SWIGExceptionHelper/ExceptionDelegate,librets.libretsPINVOKE/SWIGExceptionHelper/ExceptionDelegate,librets.libretsPINVOKE/SWIGExceptionHelper/ExceptionDelegate,librets.libretsPINVOKE/SWIGExceptionHelper/ExceptionDelegate,librets.libretsPINVOKE/SWIGExceptionHelper/ExceptionDelegate,librets.libretsPINVOKE/SWIGExceptionHelper/ExceptionDelegate,librets.libretsPINVOKE/SWIGExceptionHelper/ExceptionDelegate)
at librets.libretsPINVOKE+SWIGExceptionHelper..cctor () [0x000ef] in :0
--- End of inner exception stack trace ---
at librets.libretsPINVOKE..cctor () [0x00000] in :0
--- End of inner exception stack trace ---
at librets.RetsSession..cctor () [0x00000] in :0

--- End of inner exception stack trace ---

When I use librets to search for properties I get zero results however when I use my browser or the requests module I get 141 results. I know changes were made during the time frame provided so the 141 makes more sense and in fact includes the property I know had photos added. Other queries do get results so this seems to be an edge case but a very troubling one.

Here is a sample script which reproduces the problem:

import librets

rets_url = 'https://rets.crmls.org/contact/rets/login'
username = 'YOUR_USERNAME'
password = 'YOUR_PASSWORD'

# Fetch with librets
session = librets.RetsSession(rets_url)
session.Login(username, password)

request = session.CreateSearchRequest('Property', 'CrossProperty', '(TimestampModified=2016-04-22T15:31:00-2016-04-22T15:32:00)')
request.SetCountType(librets.SearchRequest.RECORD_COUNT_AND_RESULTS)
request.SetFormatType(librets.SearchRequest.COMPACT)

results = session.Search(request)

query = request.GetQueryString()

print 'LibRets Output:'
print '    Query: %s ' % query
print '    Count: %s' % results.GetCount() # 0
print '    Text: %s' % results.GetReplyText() # No Records Found
print '    HasNext: %s' % results.HasNext() # False
print ''

# Fetch without librets
import requests
from xml.etree import ElementTree
from requests.auth import HTTPDigestAuth

session_http = requests.get(rets_url, auth=HTTPDigestAuth(username, password))

# Use the exact query returned from librets
url = 'https://rets.crmls.org/contact/rets/search?%s' % query

data = requests.get(url, cookies=session_http.cookies).text
response = ElementTree.fromstring(data)

print 'HTTP Output:'
print '    Count: %s' % response.find('COUNT').get('Records') # 141

• Note that the count will go down over time as properties are modified again

Trying to build the librets node.js addon I get the following error:

C:\Users\xxxxxxx\libRETS\project\swig\node.js>node-gyp build
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | win32 | x64
gyp info spawn msbuild
gyp info spawn args [ 'build/binding.sln',
gyp info spawn args '/clp:Verbosity=minimal',
gyp info spawn args '/nologo',
gyp info spawn args '/p:Configuration=Release;Platform=x64' ]
Building the projects in this solution one at a time. To enable parallel build,
please add the "/m" switch.
librets_wrap.cpp
c1xx : fatal error C1083: Cannot open source file: '..\librets_wrap.cpp': No su
ch file or directory [C:\Users\rxxxxxxx\libRETS\project\swig\node.js\build\libre
ts.vcxproj]
gyp ERR! build error
gyp ERR! stack Error: msbuild failed with exit code: 1
gyp ERR! stack at ChildProcess.onExit (C:\Program Files\nodejs\node_modules
node-gyp\lib\build.js:270:23)
gyp ERR! stack at emitTwo (events.js:87:13)
gyp ERR! stack at ChildProcess.emit (events.js:172:7)
gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_proces
s.js:200:12)
gyp ERR! System Windows_NT 6.1.7601
gyp ERR! command "C:\Program Files\nodejs\node.exe" "C:\Program Files\nodej
s\node_modules\node-gyp\bin\node-gyp.js" "build"
gyp ERR! cwd C:\Users\xxxxxxx\libRETS\project\swig\node.js
gyp ERR! node -v v4.2.2
gyp ERR! node-gyp -v v3.0.3
gyp ERR! not ok

One of the MLSs we connect to changed their URL from non-secured http://matrixrets.ntreis.net/rets/login.ashx to secured https://matrixrets.ntreis.net/rets/login.ashx

Now when calling Session.Login(userName, password) an exception is thrown:

"Could not get URL [ https://matrixrets.ntreis.net/rets/login.ashx] - HTTP response code: 503"

I've tried versions 1.5.2, 1.6.1, and 1.6.2 of the libret-dotnet library and they all have the same result.

Is there something that needs to be set or called to connect to an https server?

Oddly this worked fine with 1.6.1 and we haven't changed anything, but are pulling a LOT (300 million +) photos from dozens of feeds. We started seeing these errors in logs and unsure how to interpret. I asked user group and no response so posting issue here.

rets_client.pyc: ERROR: Error [in method 'new_GetObjectRequest', argument 1 of type 'std::string'] librets.GetObjectRequest(Agent, Photo) for rec_id [259510440017078]
rets_client.pyc: DEBUG: Traceback (most recent call last):
  File "/Users/mike/Work/etl-service/etlservice/util/rets_client.py", line 919, in _get_photos
    request = librets.GetObjectRequest(ptype.decode('utf-8'), otype.decode('utf-8'))
  File "/Library/Python/2.7/site-packages/librets.py", line 1274, in __init__
    this = _librets.new_GetObjectRequest(resource, type)
TypeError: in method 'new_GetObjectRequest', argument 1 of type 'std::string'

Librets 1.6.1 on Ubuntu Linux and errors for both Property and Agent photo pulls. Any idea what and why building the request object is failing and how to diagnose this? Is it something like MLS system firewall or actual bad data type passed as argument in function?

Navid-Devs-MacBook-Pro:libRETS navid_dev$ build
-bash: build: command not found
Navid-Devs-MacBook-Pro:libRETS navid_dev$ make
antlr -o build/librets/antlr ./project/librets/src/rets-sql.g
ANTLR Parser Generator Version 2.7.7 (20171210) 1989-2005
antlr -o build/librets/antlr -glib ./project/librets/src/rets-sql.g ./project/librets/src/dmql-tree.g
ANTLR Parser Generator Version 2.7.7 (20171210) 1989-2005
antlr -o build/librets/antlr -glib ./project/librets/src/rets-sql.g ./project/librets/src/get-object-tree.g
ANTLR Parser Generator Version 2.7.7 (20171210) 1989-2005
antlr -o build/librets/antlr -glib ./project/librets/src/rets-sql.g ./project/librets/src/lookup-tree.g
ANTLR Parser Generator Version 2.7.7 (20171210) 1989-2005
antlr -o build/librets/antlr -glib ./project/librets/src/rets-sql.g ./project/librets/src/lookup-columns-tree.g
ANTLR Parser Generator Version 2.7.7 (20171210) 1989-2005
touch build/librets/antlr/.antlr-up-to-date
g++ -g -O2 -fPIC -DHAVE_CONFIG_H -DTARGET_UNIX -DLIBRETS_VERSION='"1.6.3-SNAPSHOT"' -I/usr/local/include -I./project/librets/include -c build/librets/antlr/RetsSqlLexer.cpp -o build/librets/antlr/RetsSqlLexer.o
In file included from build/librets/antlr/RetsSqlLexer.cpp:2:
build/librets/antlr/RetsSqlLexer.hpp:4:10: fatal error: 'antlr/config.hpp' file not found
#include <antlr/config.hpp>
^~~~~~~~~~~~~~~~~~
1 error generated.
make[1]: *** [build/librets/antlr/RetsSqlLexer.o] Error 1
make: *** [all] Error 2
Navid-Devs-MacBook-Pro:libRETS navid_dev$

Thanks in advance for any help

Hi,

I'm receiving the following error, sometime after reading 10k, records or 90k records, seems to occur within my loop when it tries to read the next record

Error Received:

EnhancedVOWResidentialProperty Record count: 401223

Working with Record: 79425 of 401223
Error(GrabRetsData 007): The data stream ended before the XML parser could parse everything.
This can happen if the network connection terminates before all the data has been received.

this is the code Block:

`RetsSession session = new RetsSession("http://retsau.torontomls.net:6103/rets-treb3pv/server/login");
session.SetModeFlags(RetsSession.MODE_NO_SSL_VERIFY);
Int16 iReturnCode = 0;
try
{

            session.SetTimeout(60);
            Console.WriteLine();

            if (session.Login(sUserID, sPassword))
            {

                SearchRequest searchRequest = session.CreateSearchRequest(
                                                    "Property",
                                                    sSearchClass,
                                                    sQuery);

                searchRequest.SetStandardNames(true);
                searchRequest.SetOffset(SearchRequest.OFFSET_NONE);
                searchRequest.SetCountType(SearchRequest.CountType.RECORD_COUNT_AND_RESULTS);
                searchRequest.SetFormatType(SearchRequest.FormatType.COMPACT_DECODED);
                searchRequest.SetLimit(SearchRequest.LIMIT_DEFAULT);

                SearchResultSet results = session.Search(searchRequest);

                Console.WriteLine();
                Console.WriteLine();
                Console.WriteLine();
                Console.WriteLine(sTablePrefix + sSearchClass + "\tRecord count: " + results.GetCount());
                Console.WriteLine();

                
                int iTotRecords = results.GetCount();
                int iTotRecordsProcessed = 0;


                while (results.HasNext())
                {

                        iTotRecordsProcessed++;
                        Console.Write("\rWorking with Record: " + iTotRecordsProcessed.ToString() + " of " + iTotRecords.ToString() + "\t");
                }
                appLog.WriteEntry("Finished: " + sTablePrefix + sSearchClass + "\tRecord count: " + results.GetCount(), EventLogEntryType.Information, 8);
            }

        }
        catch (Exception oException)
        {
            Console.WriteLine();
            Console.WriteLine("Error(GrabRetsData 007): " + oException.Message);
        }


        `

I'm not sure why the search is failing, not log log information available.

/libRETS/build/swig/node.js $ node search.js
Action:
Requested RETS version: RETS/1.5
Actual RETS version: RETS/1.5
Exception: Error: RetsReplyException

thanks

Brian

I'm having trouble getting a RETS 1.8.0 GetObject transaction working. The server I'm connecting to expects the identifier query parameter to be called "ID", but libRETS is passing "Identifier" instead.

I found where this is happening here:

I checked the RETS 1.8.0 specification and on page 38, under 5.3 Required Request Arguments, it says "ID", which is what the server is looking for but isn't what libRETS is sending.

Can you clarify whether I've misinterpreted the spec or whether this is a bug? Thanks much! libRETS has been very helpful to me for a few months now.

I'm working with RETS server 1.7 and attempting to perform a full database query with all fields.

All the other fields came out fine, except for Date fields. For some reason, the date value I get is a weird format which only includes minutes and seconds. The format is XX:YY.0, see example here.

I checked with my board's MLS department they are certain the field includes both date and time. So for whatever reason, librets is only getting the time in minutes and seconds.

What is the problem?

My query code below:

request = session.CreateSearchRequest(
    'Property',
    'ResidentialProperty',
    '(status="A") #status='A' returns all records
)
request.SetSelect('')
request.SetFormatType(librets.SearchRequest.COMPACT_DECODED)
request.SetLimit(librets.SearchRequest.LIMIT_NONE)
request.SetOffset(librets.SearchRequest.OFFSET_NONE)
request.SetCountType(librets.SearchRequest.RECORD_COUNT_AND_RESULTS)

results = session.Search(request)

I am running on a debian linux system and am trying to build the node.js version but I keep getting this back:

Any help would be greatly appreciated.

Is this portion of the release notes correct?

Note: there is a breaking change here in that the FORMAT defaulted to COMPACT and was optional. Now users must either explicitly set FORMAT or use the PAYLOAD option. Multiple tickets.

When I look at the code history it appears that FORMAT defaulted to COMPACT-DECODED before.

I assumed from the release note that if format wasn't specified it was COMPACT and updated the code I was converting incorrectly. I should have verified.

If it is incorrect, is there a way to update the release note.

Not sure if this is an issue or not but thought I might ask.

When I issue a GetObjectRequest using libRETS I get the following error against the RETS server I am using.

<?xml version="1.0" ?>
<RETS ReplyCode="20402" ReplyText="Invalid Identifier">
</RETS>

In talking with the admins, they are saying that my request is not setup properly. Here is what they have said...

"I can see your query and I believe I found the issue from your query. You are sending Identifier= instead of ID=, Change Identifier to ID

Resource=Property
Type=Photo
Location=0
Identifier=229375:1 - Use ID=229375:* or 1-40
"

My understanding is that creating the request as such is wrapped up into libRETS GetRequestObject and that I do not have control over this.

Is this true?

Will have to fashion my own request or is there a way I can still use libRETS give this?

I checked the RETS 1.8 spec and it pretty clearly says Identifier but it has ID crossed out on page 33...
Any insight is appreciated!

Thanks,

Ryan

import librets is producing an ImportError:

>>> import librets
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "librets.py", line 17, in <module>
    _librets = swig_import_helper()
  File "librets.py", line 16, in swig_import_helper
    return importlib.import_module('_librets')
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_librets.so, 2): Symbol not found: __ZN7librets11RetsSession11GetMetadataERSo
  Referenced from: /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_librets.so
  Expected in: flat namespace
 in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_librets.so

Here was my installation workflow:

1. Downloaded master branch of the git repo as .zip, unpacked
2. Ran ./autogen.sh
3. Ran ./configure --disable-debug --enable-shared_dependencies --prefix=/opt/local --with-boost-prefix=/opt/local --disable-dotnet --disable-java --disable-perl --disable-php --disable-ruby --disable-python3
4. Ran make, then sudo make install
5. Ran python setup.py install from build/swig/python

No idea where to go from this error, Googling returned nothing. Any help would be much appreciated!

I get this error when trying to build the test target in xcode.

python2.7
librest:1.6.1
code like that:

request = rets_session.CreateSearchRequest(res_name, cls_name, query_str)
results = rets_session.Search(request)
mc.logger.debug( "Record count: " + results.GetCount()) #value is 659

columns = results.GetColumns()
mc.logger.debug( "count: " + len(columns0) #value is 0!!!`

what's wrong with my way?
thanks!

I'm trying to use Python to sync my RETS data to a local database, it works but looks very inefficient. It seems like the record is queried and written to database multiple times. Can someone possible take a look and tell me what I have wrong?

    while results.HasNext():
        for column in columns:
            k = column
            v = results.GetString(column)
            if v is None:
                tmpdict = {"{}".format(k): "{}".format('None')}
                listing.update(tmpdict)
            elif v  == '':
                tmpdict = {"{}".format(k): "{}".format('None')}
                listing.update(tmpdict)
            else:
                tmpdict = {"{}".format(k) : "{}".format(v)}
                listing.update(tmpdict)
            table.put_item(
                Item = listing
            )

What is the best way to grab the full set and also prevent nightmares if an MLS decides to update the modified timestamp for the entire database in a single day? (Python snippet example)

Using python's binding for librets, and attempting to iterate through metadata resource/class and pull all records for that class for the initial pull, I come across a market for example with 43K records with (L_UpdateDate=1957-01-01T00:00:00+) and it will only retrieve 2500 at a time.

        request.SetLimit(librets.SearchRequest.LIMIT_DEFAULT) # changing these does nothing on some
        request.SetOffset(librets.SearchRequest.OFFSET_NONE) # changing these does nothing on some
        request.SetCountType(librets.SearchRequest.RECORD_COUNT_AND_RESULTS)
        request.SetFormatType(librets.SearchRequest.COMPACT)

        # perform query
        t_start = time.strftime('%Y-%m-%d %H:%M:%S')
        results = session.Search(request)
        record_count = results.GetCount()

        print "Record count: " + `record_count`
        print
        columns = results.GetColumns()

        while results.HasNext():
            rec = {'request-id':request_id,'data':{}}
            for column in columns:
                rec['data'][column] = results.GetString(column).decode('utf-8')  # had to fix encoding issues

As @ktgeek mentioned in other posts, all the MAY rules mean RETS server vendors don't have to guarantee cursor position or properly support the limit/offset. In some markets this is causing issues with missing some listings during larger pulls, or having to "chunk" results into many queries by last modified or price intervals, etc. Unfortunately we've come across some large markets that during conversions or major changes update the timestamp in a single day for a million+ records and wreak havoc on the last modified approach.

g++ -g -O2 -DHAVE_CONFIG_H -fPIC /Users/Ubene/Documents/librets/client/libRETS/librets-config-inplace --cflags -I./project/librets/include -Ibuild/swig/csharp -I/Users/Ubene/Documents/librets/client/libRETS/project/swig/csharp -I/Users/Ubene/Documents/librets/client/libRETS/project/swig -c build/swig/csharp/librets_wrap.cpp -o build/swig/csharp/librets_wrap.o
In file included from build/swig/csharp/librets_wrap.cpp:417:
/Users/Ubene/Documents/librets/client/libRETS/project/swig/librets_bridge.h:58:72: error: no type named 'streamsize' in namespace 'std'
std::size_t write(unsigned char buffer[], std::size_t offset, std::streamsize length) const;
~~~~~^
1 error generated.
make[1]: *** [build/swig/csharp/librets_wrap.o] Error 1

What happened here> anyone knows?

hello, i would like to report broken links to the vs 2003-2008 source code, for compile sources using visual studio, here is the link, https://github.com/NationalAssociationOfRealtors/libRETS/wiki/Building-on-Windows.

look like crt.realtor.org, is no a redirect so any prior link that use to work no work, when i try to us SNV to check out source for any of thos location in the build windows help description.

The librets-dotnet.dll is machine type COFF-x86 for the 64 bit build when it should be X64.

The latest version of LIBRETS is built using: X-Librets-Version: 1.6.2, libCURL - libcurl/7.35.0 OpenSSL/1.0.0a
Is there someone that could do a build of LIBRETS with more recent versions of LIbCURL and OpenSSL that support TLS 1.2 connections?

Hi,

Is It possible to use threads when downloading a listing's photos in c #, I've read several questions but have never come across a resolution. I'm trying to pickup the speed of my listing's download from the rets server. Any suggestions would be greatly appreciated.

Thanks in advance.

MetadataTable.DataType oDataType = table.GetDataType();
only returns type CHARACTER or DATE for tables... even though the metadata is DECIMAL or something else..

I'm using the dotnet lib within C#

http://nationalassociationofrealtors.github.io/libRETS/downloads/

doc/build/windows.txt has references to https://code.crt.realtors.org/svn/librets/ which does not seem to be available anymore.

Where can I get vendor stuff to do Windows build?

Everytime I try to use this lib, I get an Unauthorized query response back. I get logged in just fine but no matter what I try, I cannot seem to query the RETS system.

Anyone have any ideas?

I'm having an issue getting this to work on azure functions. I keep getting The type initializer for 'librets.libretsPINVOKE' threw an exception.

I have both the 32Bit and 64Bit dll's provided on nuget 1.6.2, does anyone have a solution for this?

when using librets.dotnet V4.0.30319 ( from Nuget Package 1.6.2 ) I seem to be encountering a lot of 404 connection errors. I have spoken to the rets server people and they tell me that they have no record of any connection problems. They seem to think its all on my end.. Prior I don't think that I was ever getting any errors on connection. I'm not sure if it related to the latest librets or not..
this error I'm getting is:

Could not get URL [ http://rets.torontomls.net:6103/rets-treb3pv/server/action] - HTTP response code: 401

Any help would be greatly appreciated..
thanks in advance..

You'll see it near the C# section. Seems like it was supposed to be it's own section like the other languages have. Just wanted to help. Thanks!

Link where the bug is found:
https://github.com/NationalAssociationOfRealtors/libRETS/wiki/Developer's-Guide#getobject-fetch

In C#, we will demonstrate two ways to fetch the data:

// C#
    GetObjectResponse response = session.GetObject(request);

When attempting to extend the RetsSession class in PHP, and attempting to call GetAllTables(), I ran into an issue where I got a Fatal error: No matching function for overloaded 'RetsMetadata_GetAllTables' when calling the method as

$this->GetAllTables($class);

as demonstrated in the php5/metadata.php example code file.

I was able to resolve it by changing my code to

$this->getAllTables($resource, $class);

I'm not sure if this is a SWIG problem or a libRETS problem. The code generated in librets.php looks like this for me:

function GetAllTables($metadataClass_or_resourceName,$className=null) {
    switch (func_num_args()) {
    case 1: $r=RetsMetadata_GetAllTables($this->_cPtr,$metadataClass_or_resourceName); break;
    default: $r=RetsMetadata_GetAllTables($this->_cPtr,$metadataClass_or_resourceName,$className);
    }
    if (is_resource($r)) {
        $c=substr(get_resource_type($r), (strpos(get_resource_type($r), '__') ? strpos(get_resource_type($r), '__') + 2 : 3));
        if (class_exists($c)) return new $c($r);
        return new MetadataTableList($r);
    }
    return $r;
}

I suspect the issue could be in the variable number of args passed, but I'm not really sure why passing it both parameters allows me to extend the class, but passing only the class does not.

We are trying to use librets through the PHP binding. Everything compiled without issue. However, we have noticed that as we loop through search results, memory is slowly being lost with just search and lost a lot faster when trying to fetch objects.

We believe the issue is with a reference counting problem - as valgrind shows no issues with PHP at shutdown - but something is holding onto a reference while it is running with no way to free it. Loss of memory is about 1K per 10 records.

We are running PHP 5.6 latest, swig 3.0.2, and this was compiled against librets 1.5.3 because we have boost 1.39 - options to configure:

PATH=$PATH:/usr/java/latest/bin ./configure
--enable-shared_dependencies
--disable-python
--disable-java
--disable-perl

Here is the PHP script:

<?php

require_once('librets.php');

$sURL = 'RETS URL';
$sUser = 'username';
$sPass = 'password';

function meminfo($sCheckpoint)
{
   echo 'Memory ', $sCheckpoint, ': ', round(memory_get_usage() / 1024, 2), 'KiB', \PHP_EOL;
}

function search(RetsSession $oSession, $nLimit, $nOffset = 0)
{
   echo 'Searching ... ';

   $oRequest = $oSession->CreateSearchRequest('Property', 'Listing', '(ListPrice=3000000000-)');
   $oRequest->SetLimit($nLimit);
   $oRequest->SetOffset(0);
   $oRequest->SetCountType(SearchRequest_RECORD_COUNT_AND_RESULTS);
   $oRequest->SetStandardNames(false);

   $oResults = $oSession->Search($oRequest);
   $oColumns = $oResults->GetColumns();

   $kHeader = [];
   for($n = 0; $n < $oColumns->size(); $n++) {
       $kHeader[$oColumns->get($n)] = null;
   }

   echo 'found: ', $oResults->getCount(), PHP_EOL;

   meminfo('before calling on objects');

   $nRecord = 0;
   while ($oResults->HasNext()) {

      echo 'Fetching record ', $nRecord++, ' ... ';

      $kRecord = [];
      foreach(array_keys($kHeader) as $sKey) {
         $kRecord[$sKey] = $oResults->GetString($sKey);
      }

      if (!empty($kRecord['Matrix_Unique_ID'])) {

         $oImageRequest = new GetObjectRequest('Property', 'Photo');
         $oImageRequest->AddAllObjects($kRecord['Matrix_Unique_ID']);
         $oImageResults = $oSession->GetObject($oImageRequest);
         while ($oImageObj = $oImageResults->NextObject()) {
             file_put_contents('/tmp/out', $oImageObj->GetDataAsString());
         }

      }

      meminfo('after loading images');

   }

}

function run($sURL, $sUser, $sPass, $nLimit)
{
   $oSession = new \RetsSession($sURL);
   if (!$oSession->Login($sUser, $sPass)) {
      die("Login Failed" . PHP_EOL);
   }

   echo 'RETS Version: ', $oSession->RetsVersionToString($oSession->GetDetectedRetsVersion()), PHP_EOL;

   meminfo('before searching');
   if ($nLimit > 10) {

      for($n = 0; $n < $nLimit / 10; $n++) {
         search($oSession, 10, $n);
      }

   } else {

      search($oSession, $nLimit);

   }
   meminfo('after searching');

   $oSession->Logout();
}

meminfo('before RETS Session');
run($sURL, $sUser, $sPass, empty($argv[1]) ? 5 : $argv[1]);
meminfo('after RETS Session');

Please help us track down the memory leak - thanks!

Are there any plans to implement support for the RETS 1.8 Update transaction?
Specifically the inclusion of the required Action request argument

Hello,

I am unable to log in to CCRM, a Rapattoni feed, from librets. I have checked the credentials by successfully logging in with PHRETS version 1. In the code below, I have set the user agent and rets version after the session is created, prior to calling the Login method, as described in the FAQ and other documentation. A log file has been attached as well, that may point out the problem.

What I noticed in the difference between the successful login for PHRETS and the unsuccessful login is the connection number. In the successful login, the second login attempt (needed for hand-shaking with digest authentication), the connection is made to connection 0, that was created on the first login attempt.

In the librets.log attached below, the second, third, and fourth attempts all use an incremental connection number (connection 1, 2, 3) until the Login method gives up. It seems that it can't complete the authentication because it isn't trying to connect back to connection #0 with the correct authorization and session cookie. To clarify, it sends the authorization header, but not the cookie and it makes a new connection.

Has anyone run into this problem? I've searched the issues here and on the librets forum, but can't figure out how to solve this problem.

thanks!

Sample code :

 using System;
 using librets;

 namespace TestLibrets
 {
     class Program
     {
        static void Main(string[] args)
        {
            RetsSession session = null;
            try
            {
                session = new RetsSession("LoginUrl"); // shortened for readability
                session.SetRetsVersion(RetsVersion.RETS_1_7_2);
                session.SetUserAgent("Aumnia/1.0");
                session.SetHttpLogName("librets.log");

                string user = "User";
                string passwd = "Password";
                if (!session.Login(user, passwd))
                {
                    Console.WriteLine("Invalid login");
                    Environment.Exit(0);
                }

                Console.WriteLine(".Net version: " + System.Environment.Version);

                LoginResponse login = session.GetLoginResponse();
                Console.WriteLine("Member name: " + login.GetMemberName());

                CapabilityUrls capabilityUrls = session.GetCapabilityUrls();
                Console.WriteLine("Search URL: " + capabilityUrls.GetSearchUrl());

                LogoutResponse logout = session.Logout();
                Console.WriteLine("Billing info: " + logout.GetBillingInfo());
                Console.WriteLine("Logout message: " + logout.GetLogoutMessage());
                Console.WriteLine("Connect time: " + logout.GetConnectTime());

            }
            finally
            {
                // Be sure to dispose RetsSession when finished, otherwise the
                // TextWriter Dispose() method may be called prior to RetsSession.
                if (session != null)
                {
                    session.Dispose();
                }

                Console.WriteLine("Enter key to Exit");
                Console.ReadLine();
            }

         }
     }
 }

Log:

* Hostname was NOT found in DNS cache
*   Trying 72.26.113.170...
* Connected to rets172lax.raprets.com (72.26.113.170) port 6103 (#0)

>>> Sent
GET /CentralCoast/CCRM/login.aspx HTTP/1.1

Host: rets172lax.raprets.com:6103

Accept: */*

RETS-Version: RETS/1.7.2

User-Agent: Aumnia/1.0

X-Librets-Version: 1.6.2, libCURL - libcurl/7.35.0 OpenSSL/1.0.0a

<<< Received
HTTP/1.1 401 Unauthorized. Not Authenticated..

* Server Microsoft-IIS/8.5 is not blacklisted

<<< Received
Server: Microsoft-IIS/8.5

X-AspNet-Version: 2.0.50727

WWW-Authenticate: Digest realm="CCRM", nonce="783c6fdfa68fcc076a8674b553bc19d1", opaque="c43c55b565d8e2ef3274c8b81ecd1236", qop="auth"

Vary: Accept-Encoding

Cache-Control: private

Content-Type: text/xml

Date: Fri, 02 Sep 2016 16:50:54 GMT

RETS-Version: RETS/1.7.2

Transfer-Encoding: chunked

Connection: Keep-Alive

X-Powered-By: ASP.NET (203)

* Ignoring the response-body

<<< Received
0

* Connection #0 to host rets172lax.raprets.com left intact
* Issue another request to this URL: 'http://rets172lax.raprets.com:6103/CentralCoast/CCRM/login.aspx'
* Found bundle for host rets172lax.raprets.com: 0x6546b28

* Hostname was found in DNS cache
*   Trying 72.26.113.170...
* Connected to rets172lax.raprets.com (72.26.113.170) port 6103 (#1)
* Server auth using Digest with user 'XXXX'

>>> Sent
GET /CentralCoast/CCRM/login.aspx HTTP/1.1

Authorization: Digest username="XXXX", realm="CCRM", nonce="783c6fdfa68fcc076a8674b553bc19d1", uri="/CentralCoast/CCRM/login.aspx", cnonce="ODRkYWRiOGY2ZWY0Njg0MzAwMDhlOTA2MDAwMDAwMDA=", nc=00000001, qop=auth, response="0c69958b31411b71b9c5688135a7e9da", opaque="c43c55b565d8e2ef3274c8b81ecd1236"

Host: rets172lax.raprets.com:6103

Accept: */*

RETS-Version: RETS/1.7.2

User-Agent: Aumnia/1.0

X-Librets-Version: 1.6.2, libCURL - libcurl/7.35.0 OpenSSL/1.0.0a
<<< Received
HTTP/1.1 401 Bad Request. Response did not match. /CentralCoast/CCRM/login.aspx :client

* Server Microsoft-IIS/8.5 is not blacklisted

<<< Received
Server: Microsoft-IIS/8.5

X-AspNet-Version: 2.0.50727

* Authentication problem. Ignoring this.

<<< Received
WWW-Authenticate: Digest realm="CCRM", nonce="783c6fdfa68fcc076a8674b553bc19d1", opaque="c43c55b565d8e2ef3274c8b81ecd1236", qop="auth"

Vary: Accept-Encoding

Cache-Control: private

Content-Type: text/xml

Date: Fri, 02 Sep 2016 16:50:55 GMT

RETS-Version: RETS/1.7.2

Transfer-Encoding: chunked

Connection: Keep-Alive

X-Powered-By: ASP.NET (204)
0

* Connection #1 to host rets172lax.raprets.com left intact
* Found bundle for host rets172lax.raprets.com: 0x6546b28
* Hostname was found in DNS cache
*   Trying 72.26.113.170...
* Connected to rets172lax.raprets.com (72.26.113.170) port 6103 (#2)
* Server auth using Digest with user 'XXXX'

>>> Sent GET /CentralCoast/CCRM/login.aspx HTTP/1.1

Authorization: Digest username="XXXX", realm="CCRM", nonce="783c6fdfa68fcc076a8674b553bc19d1", uri="/CentralCoast/CCRM/login.aspx", cnonce="YzI3Mjc5NGZkMTQwYjFlNDAwMDhlOTA3MDAwMWE5Yzg=", nc=00000001, qop=auth, response="83121c3edbd93c38a014cc24bc0d0d84", opaque="c43c55b565d8e2ef3274c8b81ecd1236"

Host: rets172lax.raprets.com:6103

Accept: */*

RETS-Version: RETS/1.7.2

User-Agent: Aumnia/1.0



X-Librets-Version: 1.6.2, libCURL - libcurl/7.35.0 OpenSSL/1.0.0a

<<< Received
HTTP/1.1 401 Bad Request. Response did not match. /CentralCoast/CCRM/login.aspx :client

* Server Microsoft-IIS/8.5 is not blacklisted

<<< Received
Server: Microsoft-IIS/8.5

X-AspNet-Version: 2.0.50727

* Ignoring duplicate digest auth header.

<<< Received
WWW-Authenticate: Digest realm="CCRM", nonce="783c6fdfa68fcc076a8674b553bc19d1", opaque="c43c55b565d8e2ef3274c8b81ecd1236", qop="auth"

Vary: Accept-Encoding

Cache-Control: private

Content-Type: text/xml

Date: Fri, 02 Sep 2016 16:50:54 GMT

RETS-Version: RETS/1.7.2

Transfer-Encoding: chunked

Connection: Keep-Alive

X-Powered-By: ASP.NET (201)



* Ignoring the response-body

<<< Received
0

* Connection #2 to host rets172lax.raprets.com left intact
* Issue another request to this URL: 'http://rets172lax.raprets.com:6103/CentralCoast/CCRM/login.aspx'
* Found bundle for host rets172lax.raprets.com: 0x6546b28
* Hostname was found in DNS cache
*   Trying 72.26.113.170...
* Connected to rets172lax.raprets.com (72.26.113.170) port 6103 (#3)
* Server auth using Digest with user 'XXXX'

>>> Sent
GET /CentralCoast/CCRM/login.aspx HTTP/1.1

Authorization: Digest username="XXXX", realm="CCRM", nonce="783c6fdfa68fcc076a8674b553bc19d1", uri="/CentralCoast/CCRM/login.aspx", cnonce="YzI3Mjc5NGZkMTQwYjFlNDAwMDhlOTA3MDAwMWE5Yzg=", nc=00000002, qop=auth, response="d428ecfce14ab4da038534bbd935b796", opaque="c43c55b565d8e2ef3274c8b81ecd1236"

Host: rets172lax.raprets.com:6103

Accept: */*

RETS-Version: RETS/1.7.2

User-Agent: Aumnia/1.0

X-Librets-Version: 1.6.2, libCURL - libcurl/7.35.0 OpenSSL/1.0.0a

<<< Received
HTTP/1.1 401 Bad Request. Response did not match. /CentralCoast/CCRM/login.aspx :client

* Server Microsoft-IIS/8.5 is not blacklisted

<<< Received
Server: Microsoft-IIS/8.5

X-AspNet-Version: 2.0.50727

* Authentication problem. Ignoring this.

<<< Received
WWW-Authenticate: Digest realm="CCRM", nonce="783c6fdfa68fcc076a8674b553bc19d1", opaque="c43c55b565d8e2ef3274c8b81ecd1236", qop="auth"

Vary: Accept-Encoding

Cache-Control: private

Content-Type: text/xml

Date: Fri, 02 Sep 2016 16:50:55 GMT

RETS-Version: RETS/1.7.2

Transfer-Encoding: chunked

Connection: Keep-Alive

X-Powered-By: ASP.NET (204)

0

* Connection #3 to host rets172lax.raprets.com left intact

Hello I'm perform to call the GetObject after doing search for multiple listing. If I only have one search element to process the GetObject work, anymore it fails.

console.log("Action: " + session.GetAction());
console.log("Requested RETS version: " + session.RetsVersionToString(session.GetRetsVersion()));
console.log("Actual RETS version: " + session.RetsVersionToString(session.GetDetectedRetsVersion()));

var request = session.CreateSearchRequest("Property", "RES", "(ListPrice=300000-)");
//var request = session.CreateSearchRequest("Property", "RES", "(MLSNUM=523829)");

request.SetStandardNames(false);
request.SetSelect("ListPrice,Bedrooms,PHOTO_COUNT,CityRets,VTourUrl,MLSNUM,PictureId,Latitude,Longitude,ListAgentEmail,LEGALDESC,StreetName,Province,PROPCODE");
request.SetLimit(librets.SearchRequest.LIMIT_DEFAULT);
request.SetOffset(librets.SearchRequest.OFFSET_NONE);
request.SetCountType(librets.SearchRequest.RECORD_COUNT_AND_RESULTS);
request.SetFormatType(librets.SearchRequest.COMPACT);
results = session.Search(request);

console.log("exports.create = {");
console.log("   Residential: [");

var first = true;

while (results.HasNext())
{
     // Gets the large image
    var get_object_request = new librets.GetObjectRequest("Property", "Photo");

    var locationPhotoUrls = [];
    var locationThumbUrls = [];

    var pictureId = results.GetString("PictureId");
    get_object_request.AddAllObjects(pictureId);
    get_object_request.SetLocation(true);

   // Rets exception happens here
    var get_object_response = session.GetObject(get_object_request);

    var object_descriptor = get_object_response.NextObject();

    // Uncomment to write all images to cloud.
    while (object_descriptor)
    {
        locationPhotoUrls.push(object_descriptor.GetLocationUrl());
        object_descriptor = get_object_response.NextObject();
    }

    get_object_request = new librets.GetObjectRequest("Property", "Thumbnail");

    get_object_request.AddAllObjects(pictureId);
    get_object_request.SetLocation(true);

    var get_object_response = session.GetObject(get_object_request);

    var object_descriptor = get_object_response.NextObject();

    // Uncomment to write all images to cloud.
    while (object_descriptor)
    {
        locationThumbUrls.push(object_descriptor.GetLocationUrl());
        object_descriptor = get_object_response.NextObject();
    }

    console.log("       {");
    console.log(
        "           'MLSNUM': '"+ results.GetString("MLSNUM") + "'" +
        ", 'LocationPhotoUrls': '" + locationPhotoUrls.toString() + "'" +
        ", 'LocationThumbUrls': '" + locationPhotoUrls.toString() + "'" + 
        ", 'ListPrice': '" + results.GetString("ListPrice") + "'" +
        ",'Bedrooms': '" + results.GetString("Bedrooms") + "'" +
        ",'PictureID': '" + results.GetString("PictureId") + "'" +
        ",'Longitude': '" + results.GetString("Longitude") + "'" +
        ",'Latitude': '" + results.GetString("Latitude") + "'" +
        ",'ListAgentEmail': '" + results.GetString("ListAgentEmail") + "'" +
        ",'PHOTO_COUNT': '" + results.GetString("PHOTO_COUNT") + "'" +
        ",'CityRets': '" + results.GetString("CityRets") + "'" +
        ",'VTourUrl': '" + results.GetString("VTourUrl") + "'" +
        ",'PROPCODE': '" + results.GetString("PROPCODE") + "'" +
        ",'StreetName': '" + results.GetString("StreetName") + "'" +
        ",'Province': '" + results.GetString("Province") + "'"
    );
    console.log("       }");
}
console.log("   ]");
console.log("};");

Would anyone be able to give me suggestion on how to a create npm module so I can use in in my node.js application?

thanks

Brian

Similar to other install documentation issue, the Homebrew formula for OSX don't work. We had a custom with boost 1.49 but has been failing last two OS updates, and URLs were wrong. We fixed our Ubuntu install scripts with correct URLs, etc. but Homebrew not working?

I edited one and still failing and appreciate any ideas on how to get Homebrew to install librets on OSX (supporting Java and Python bindings):

require 'formula'

class Librets < Formula
  homepage 'http://code.crt.realtors.org/projects/librets'
  url 'https://github.com/NationalAssociationOfRealtors/libRETS/archive/1.6.1.tar.gz'
  sha1 '0078524e12ced0ac98c472c8a9e69c8d12d24546'

  depends_on 'swig'
  depends_on 'boost'

  def install
    system "./configure", "--disable-debug",
                          "--enable-shared_dependencies",
                          "--prefix=#{prefix}",
                          "--disable-dotnet",
                          "--disable-perl",
                          "--disable-php",
                          "--disable-ruby"
    system "make install"
  end
end

This fails and unsure why. Boost installs (not the 1.49 like 1.5.3 dependencies, but 1.56.0 installs).

This is sort of a weird use case, but the RETS server I'm building toward serves up the same image for index 0 and index 1. I'd like to AddAllObjects starting at index 1 so that I don't waste time downloading the 0th image.

Right now I'm working around this by not calling GetDataAsBytes on the first result, but this is confusing (trying to reuse code across many RETS servers and this is an exception) and it'd be a lot cleaner to start at a specific index.

If there's a way I can already accomplish this, please let me know. Thanks!

I am trying to us the C++ includes in a Visual Studio 2015 project, I have downloaded Boost 1.60, unzipped to C:\boost_160, and ran to the b2 commands to populate staging/libs, in visual studio I have added the librets include directory in additional includes under c/c++ and I have added the boost/staging/libs directory under Linker additional library directories, but when I try a simple login log off command, I get an error like this
Severity Code Description Project File Line Suppression State
Error LNK2019 unresolved external symbol "public: bool __cdecl librets::RetsSession::Login(class std::basic_string<char,struct std::char_traits,class std::allocator >,class std::basic_string<char,struct std::char_traits,class std::allocator >)" (?Login@RetsSession@librets@@QEAA_NV?$basic_string@DU?$char_traits@D@std@@v?$allocator@D@2@@std@@0@Z) referenced in function main RETSTest C:\Users\Lance\Documents\Visual Studio 2015\Projects\RETSTest\RETSTest\main.obj 1

Any help would be appreciated to getting this to compile

1. Why OPENSSL_CFLAGS/LIBS is required? libcurl can be built with WinSSL (and is built with WinSSL on Windows by default)

2. Why not use auto-linking for boost libs (when .lib files are detected by compiler automatically). BOOST_XXX_LIB and all the related defines can be safely removed

Hi all, am using libRETS for the first time, and used the Developer's Guide as a tutorial, the problem is that I can't get the columns from the queries, for example:

req = session.CreateSearchRequest("Property", "CMF", "(YearBuilt=1+) , (ListingStatus=|1,2)")
res = session.Search(req)

res.GetCount() yields 438 hits, but res.GetColumns() returns an empty tuple, what am I missing?

I am using the Python binding for this.

Thanks,
Giovanni

Currently libRETS builds for static linking. This is causing more issues on certain 64 bit platforms that do not build their static libraries for position independent code.

It looks like the NEWS was updated over a year ago with improvements such as 1.8 GetObject improvements.

Any chance a new release is forthcoming?

Also, the README.md for this project seems to have gotten updated with some libRETS Docker info, which seems strange and wrong.

I see in .gitignore file you are ignoring /configure but fore some reason this may also be omitting the .configure file which is missing when downloading source tarball distributions.

Has the build process changed or is this simply an omission? If latter, please modify the .gitignore and add/push the .configure file and update tarball distribution.

• apologies if odd inquiry but prior distributions included .configure and I believe was part of distro and not some autoconf lib on server.

I was able to build librets on OSX El Captain, referencing the expat and boost library paths with ./configure, then adding -m32 to clang and ldflags in the makefile. I built librets with sudo make, then sudo make install, everything looks fine no errors I have a file librets.node in my build/Release directory how do I install this as a global npm module?

There used to be a place to download the compiled binaries. But I cannot find it. Something has happened and my LibRETS queries suddenly stopped working. Perhaps I need an update to libRETS but I cannot find the dlls.

I noticed the same thing was reported for windows links and I understand the servers are down.
But Im wondering is there any alternative locations I can access API documentations? Seems like there is currently no resources for help if something goes wrong.
Any sort of minute API documentation would be very helpful

Hello all,

I'm not sure this project is being supported but I figured I'd try asking here.

I've been trying to get librets to work on a bare installation of Debian (wheezy) but my limited knowledge of C and python for that matter are making it hard. Here's how I'm setting this up:

FROM debian:7.8

WORKDIR /root

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update -qq && apt-get upgrade -qq -y
RUN apt-get install -y apt-utils
RUN echo 'America/Los_Angeles' > /etc/timezone && dpkg-reconfigure tzdata
RUN apt-get install -y --fix-missing \
        build-essential \
        libboost-all-dev \
        curl \
        libcurl4-gnutls-dev \
        automake \
        autoconf \
        libtool \
        xsltproc \
        libexpat-dev \
        expat \
        default-jdk \
        wget \
        git \
        antlr \
        swig \
        python-dev

ENV CLASSPATH=".:/usr/share/java/antlr.jar:$CLASSPATH"
RUN alias antlr4='java -jar /usr/share/java/antlr.jar' && \
    alias grun='java org.antlr.v4.runtime.misc.TestRig'


RUN git clone https://github.com/NationalAssociationOfRealtors/libRETS.git && \
    cd libRETS/ && \
    ./autogen.sh && \
    ./configure \
        --prefix=/usr/bin \
        --disable-perl \
        --disable-ruby \
        --disable-java \
        --disable-dotnet \
        --disable-php \
        --disable-python3 \
        --enable-examples && \
    make && \
    make install

Now, for the purposes of this test I'm working under the /root directory so I end up with /root/libRETS whereas my test script is at /app/test.py and, for now simply contains:

# python
import sys
import librets

python /app/test.py, not surprisingly, exits because it can't find librets.

I've tried running python setup.py install from within /root/libRETS/project/swig/python/ but that fails yielding:

sys.platform: linux2
running install
running build
running build_py
file librets.py (for module librets) not found
file librets.py (for module librets) not found
running build_ext
building '_librets' extension
creating build
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/root
creating build/temp.linux-x86_64-2.7/root/libRETS
creating build/temp.linux-x86_64-2.7/root/libRETS/project
creating build/temp.linux-x86_64-2.7/root/libRETS/project/swig
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c librets_wrap.cpp -o build/temp.linux-x86_64-2.7/librets_wrap.o -I/root/libRETS/project/librets/include -I/usr/include -I. -fPIC
gcc: error: librets_wrap.cpp: No such file or directory
gcc: fatal error: no input files
compilation terminated.
error: command 'gcc' failed with exit status 4

Can anyone shed some light on this? I'm afraid I can't figure this out quickly enough to be useful for me.

Thank you!

I'm pulling data from multiple MLSs and have recently updated my Python app to handle each MLS in a separate thread. Some of the MLSs can be very sluggish, so I was hoping to speed things up by doing each MLS in parallel. The problem I've run into is that once any thread has initiated a RETS Search request (or GetObject request), no other thread can start a request until the first thread's request has returned.

I understand not wanting to send simultaneous requests to a single MLS, but these are independent MLSs, so I'm only trying to send a single request to any given MLS at at time.

Is there a way to avoid the blocking for requests on a different thread? Or is there a better way to
handle this situation?

Thanks,
Matt