Giter VIP home page Giter VIP logo

mapguide-api-bindings's Introduction

mapguide-api-bindings

IMPORTANT Development work on this repository has been suspended and development work is continuing on the vanilla_swig branch of MapGuide

Windows Build status

Language bindings for the MapGuide API

Motivation

We currently use a heavily modified version of SWIG to generate language bindings for the MapGuide API

This modified version of SWIG is extermely old and has an unclear audit trail of modifications which makes it difficult for us to expand language support beyond what we currently support:

  • PHP (5.x)
  • Java
  • .net (Full Framework)

Supported Platforms

Our current focus of this project is to use the current version of SWIG (3.0.12 as of writing) to generate MapGuide API bindings to support the following languages/platforms:

Platform Binding Notes Windows (x86) Windows (x64) Linux (x86) Linux (x64)
.net Binding Notes Yes Yes No (^1) Yes (^2)
PHP 7.1 Binding Notes TBD (^3) Yes (^3) TBD (^3) TBD (^3)
Java Binding Notes TBD Yes Yes (^4) Yes (^4)

(^1): Microsoft does not offer .net Core for 32-bit Linux.

(^2): Ubuntu 14.04 64-bit only

(^3): Because official MapGuide release does not (yet) bundle PHP 7.x, the current way to use this binding is through a standalone install of PHP 7.x and manually registering the MapGuide API extension with it.

(^4): Only supported on Linux distros where we provide MapGuide binaries for:

  • Ubuntu 14.04
  • CentOS 6.x

With the possibility in the future for experimental (a.k.a Use at your own risk) support for other platforms that a current and unmodified SWIG can offer us:

  • Ruby
  • Python
  • node.js
  • and much more!

Build requirements (Windows)

  • Microsoft Visual C++ 2015 (You can use the Community Edition)
    • If you have VS 2017, make sure it has the MSVC 2015 (v140) toolset installed
  • SWIG 3.0.12 (On Linux, envsetupsdk.sh can download and install this for you)
  • .net Core SDK 2.1
  • 7-zip
  • Java SDK
  • ant
  • ant-contrib
  • CMake (>= 2.8)

Build requirements (Linux)

  • Ubuntu 14.04 or CentOS 6.x
  • MapGuide is installed
  • .net Core SDK
  • Java SDK
  • dos2unix
  • p7zip (for CentOS, you need to enable EPEL repositories)
  • CMake (>= 2.8)

Before you build

You will need a pre-compiled "buildpack". This contains the minimum set of headers/libs/dlls from MapGuide needed to build the SWIG bindings. Grab the appropriate buildpacks here (URL TBD) and extract them a versioned directory under the "sdk" directory.

For example, if you are installing the 3.1 buildpack, extract the buildpack contents to sdk\3.1

On Linux, only the headers from this buildpack are needed as this project will link against the libraries of your MapGuide installation.

Build Instructions (Windows)

Build Steps (CMake)

  1. Run build_tools.cmd to build all the internal tools required for the rest of the build
  2. Run envsetup.cmd $VERSION_MAJOR $VERSION_MINOR $VERSION_BUILD $VERSION_REV <path to swig installation>. For example, if building against MGOS 3.1.1 and SWIG is installed in C:\swigwin-3.0.12, you would run envsetup.cmd 3 1 1 9378 C:\swigwin-3.0.12
  3. Run cmake_build.cmd <path to working directory> to build the SWIG bindings and associated wrappers
  4. To test any of the bindings, run:
  • For .net Core: test_dotnet_core.cmd
  • For .net Full Framework: test_dotnet_full.cmd
  • For PHP: test_php.cmd
  • For Java: test_java.cmd

Build Steps (MSBuild)

  1. Run build_tools.cmd to build all the internal tools required for the rest of the build
  2. Run envsetup.cmd $VERSION_MAJOR $VERSION_MINOR $VERSION_BUILD $VERSION_REV <path to swig installation>. For example, if building against MGOS 3.1.1 and SWIG is installed in C:\swigwin-3.0.12, you would run envsetup.cmd 3 1 1 9378 C:\swigwin-3.0.12
  3. Run build.cmd to build the SWIG bindings and associated wrappers
  4. To test any of the bindings, run:
  • For .net Core: test_dotnet_core.cmd
  • For .net Full Framework: test_dotnet_full.cmd
  • For PHP: test_php.cmd
  • For Java: test_java.cmd

Build Instructions (Linux, bare metal)

Before you start

Also note that this build process on Linux will only build the SWIG glue library for .net/Java/PHP. The mananged wrapper libraries themselves are expected to be built on Windows

Build Steps

  1. Run ./build_tools.sh to build all the internal tools required for the rest of the build
  2. Run ./envsetupsdk.sh --version $VERSION_MAJOR.$VERSION_MINOR.$VERSION_BUILD.$VERSION_REV [--with-java] [--with-dotnet] [--with-php] to prepare the environment to build for .net/Java/PHP. This will download SWIG and the equivalent MapGuide buildpack if required. Please observe the above support matrix when deciding what languages to enable. For example, --with-dotnet is useless on 32-bit Linux because there is no .net Core SDK for 32-bit Linux.
  3. Run ./cmake_build.sh --version $VERSION_MAJOR.$VERSION_MINOR.$VERSION_BUILD.$VERSION_REV --working-dir <path to working directory> [--with-java] [--with-dotnet] [--with-php] to build the glue libraries in <working directory>
  4. To test any of the bindings, run:
  • For .net Core: test_dotnet.sh
  • For Java: test_java.sh

Build Instructions (Linux, via vagrant on Windows host)

Vagrantfiles are provided which can build the required SWIG glue libraries for the supported distros on Linux.

These Vagrantfiles assume vagrant base boxes that are already present:

  • env-centos6-amd64
  • env-centos6-i386
  • env-ubuntu14-amd64
  • env-ubuntu14-i386

If these base boxes aren't present, you can build them using using packer and the provided templates in linux_build/packer. For convenience, a make_boxes.cmd wrapper batch file is included to build all these vagrant base boxes for you (make sure packer is in your Windows PATH).

  1. Enter any of the following directories:
  • linux_build/CentOS6_x64
  • linux_build/CentOS6_x86
  • linux_build/Ubuntu14_x64
  • linux_build/Ubuntu14_x86
  1. Run vagrant up to spin up the Linux VM and build the SWIG glue libraries supported on that distro. Upon completion, the glue libraries will be in (on your host):
  • Java: packages/Java/Release/$CPU/$DISTRO/libMapGuideJavaApi.so
  • .net: src/Managed/DotNet/MapGuideDotNetApi/runtimes/ubuntu14.04-x64/native/libMapGuideDotNetUnmanagedApi.so (Ubuntu 14.04 64-bit only)

mapguide-api-bindings's People

Contributors

jumpinjackie avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

mapguide-api-bindings's Issues

Portable buildpacks

It's going to be impractical to demand a full MapGuide source tree that is built in x86 and x64 configurations as a pre-requistite to build these bindings, so we should add some scripts that help collect all the necessary artifacts (headers/libs/dlls) from an existing full source tree into a portable buildpack and have our bindings built against that instead.

Automatic class id generation

  • Update preparation tools to properly rewrite class id include paths in SWIG parameter file (instead of commenting them out)
  • Run SWIG in XML mode to generate the XML dump of the MapGuide API surface
  • Update the ClassMapGen tool to take the SWIG XML dump file and scan all the m_cls_id members for their values. These values will most likely be arithmetic expressions, so the actual id will need to be evaluated (maybe use exprtk to do this)Auto-generate the class map templates from this collected data.

MGOS 3.0 target

Set up environment to allow these bindings to be built against MGOS 3.0

Investigate possibility of splitting .net assemblies

Although the original intent was to keep a monolithic assembly, it does complicate things if we want to introduce mg-desktop (that builds on top of the same common core) into the mix, which hampers our re-usability story which would be restricted down to source-compatibility.

So despite the initial rejection of preserving the split-assembly idea, we should at least try to see if it is possible at a minimum to split off the (Foundation, Geometry, PlatformBase) parts of the MapGuide API and have the (MapGuideCommon, WebSupport, WebApp) part depend on it in another assembly.

Travis/AppVeyor setup

Now that our .net binding and its test suite are both functional, let's keep it that way by hooking the test suite up to Travis/AppVeyor

[.net] Target netstandard

Once #19 is fixed, the first port of call is for this to target netstandard. As our SWIG wrapper uses a very low surface area of .net in terms of classes, we should be able to target the lowest netstandard1.0. This means our wrapper will work on the oldest possible versions of .net core and .net full framework.

Master class map definition

Currently we are maintaining separate (but identical) class maps for .net and PHP.

This class map is a dictionary of class id -> class name, which is used to workaround SWIG's base inability to handle downcasting of derived types.

This class map should ideally be a JSON file that's fed to a tool that outputs what is currently PhpClassMap.cpp and MgClassMap.cs source files. This way we only have one source of truth.

Try using char* for marshaling strings

We are currently having an undesirable effect whereby our marshaled wchar_t strings in .net core on Linux have garbage trailing characters. This blows up a fair chunk of our test suite on Linux.

Although our wchar_t typemap is based on the official Java binding, we lack the ability to infer the length of input strings, which may explain why we get trailing garbage.

Instead, we should probably try using char instead of wchar_t.

char has a consistent size on both Windows and Linux and the official PHP binding already uses char for strings that travel to/from the MapGuide module boundary, so it's not like char is an untested option.

Version stamping tool

Because we intend to build these bindings for more than one version of MapGuide, we need a tool to stamp the appropriate versions for source files that contain version numbers

PHP 7 binding

Version 3.0.11 was recently released which added long-awaited support for PHP 7.

As it is imperative that we escape the pending EOL of PHP 5.6 which is in less than 2 years time, we need to explore a usable path where we can bundle in PHP 7 and a MapGuide API binding that works with this version.

Current checklist:

  • Get downcasting to work. For example: MgSiteConnection::CreateService returns a proxy of MgService instead of the appropriate derived class
  • Test APIs that deal with MgByteReader to verify our byte typemaps are working
  • Because we downgraded to PHP 7.1, check if the pre-build sed patch workaround is still required
  • Make sure we can throw MgException based exceptions
  • Make sure MgException based exceptions are being properly released
  • We should be building for the NTS (non-thread-safe) profile not TS as the binding in its current form (which is built for thread-safe PHP) cannot be used if hosting PHP in IIS (it uses non-thread-safe PHP). We can also use non-thread-safe PHP in Apache via mod_fcgid, so from a practical standpoint it's simpler to just build for the NTS profile.
  • Since we've monkey-patched a custom destructor for MgException-derived proxy classes, find a way to get SWIG to not generate all those delete functions that will never get called.
  • Put all the generated PHP code under a OSGeo\MapGuide namespace
  • Get it building (Linux)
  • Make existing PHP test suite pass (Windows)
  • Make existing PHP test suite pass (Linux)

.net Binding (remaining checklist)

  • Verify nuget package is consumable on full .net framework without any additional changes
    • Add MSBuild targets file for full framework to ensure unmanaged dlls are being copied to output
    • Have MapGuideDotNetApi build for AnyCPU
  • Add full framework version of entire .net test suite
  • Ensure it builds (Linux)
  • Make .net test suite pass (Linux)
    • String parameters seem to be completed discarded or trashed (eg. new MgUserInformation("username", "password") throws MgInvalidArgumentException due to empty string parameters)

Rename SWIG glue library to (lib)MapGuideDotNetUnmanagedApi

To simplify #12 the SWIG glue library that we currently make (MapGuideDotNetCoreUnmanagedApi.dll) should be re-usable for the full .net framework binding.

Therefore the name MapGuideDotNet_Core_UnmanagedApi is a bit of a misnomer as there isn't really any CoreCLR-specific native code in this library and should be renamed to MapGuideDotNetUnmanagedApi instead

Java binding

  • Get it building (Windows)
  • Get it building (Linux)
  • Build should follow same pattern as .net. Only build the SWIG glue library. Build the Java portion as its own step.
  • Ensure java test suite passes (Windows)
  • CentOS: Need to find and manually link in libmawt.so as linking in JNI_LIBRARIES pulls in this dependency causing an UnsatisfiedLinkError (not necessary, just don't link JNI_LIBRARIES)
  • Ensure java test suite passes (Linux)
    • Ubuntu 14.04 (64-bit)
    • Ubuntu 14.04 (32-bit)
    • CentOS 6 (64-bit)
    • CentOS 6 (32-bit)
  • Rename delete SWIG cleanup proxy method to something else (SWIG lets you choose a different name)

Improve test logging

Currently our test suite can run on Linux with segfaulting (yay!) but still has lots of test failures. Looking at the test log show little that can help us because the parameters that are logged before each test are their values before any cleaning and normalization is done to them.

We need to restructure our test execution so that there is an explicit parameter collection and sanitization step that allows our logging mechanism to log the actual (post-sanitization) parameters that are used to execute the operation

Rewrite preparation tools in C++

Sadly, we cannot rely on .net core as a universally available dependency as it is not available on 32-bit Linux and it is not available for CentOS 6.x 64-bit.

Fortunately, these tools (SwigPrepare, PhpPostProcess) don't do overly complex things so it should be easy to rewrite them in C++

Check that error messages are being read

I've been noticing that the sample apps I'm building with the new MapGuideDotNetApi wrapper that any unhandled exceptions being thrown don't appear to have any error messages.

Check that there is no issues with exception messages being trashed when crossing the .net <-> native boundary.

Re-instate xerces headers in buildpack

The out string typemaps for the .net SWIG glue library will not compile on Linux without xerces headers present. This has slipped under my radar because the test Linux VM I was building these bindings in already had libxerces-c-dev package installed, which includes headers.

Package buildpack headers for Linux build

The work in #15 allows us to produce portable windows buildpacks so we can build this binding on Windows in both x86 and x64 without needing a full MapGuide source tree to build against.

For Linux, we should be able to build against an existing MapGuide install, as Linux shared libraries are both the "dll" and the "import .lib" in this case, so the missing piece that allows us to build our SWIG bindings is the necessary header files, which are not part of a MapGuide install.

We can just repurpose existing headers from the portable windows buildpack for this purpose.

Hook up TravisCI/AppVeyor

AppVeyor:

  • Use VS 2017 image (hope it has v140 toolset, otherwise fallback to VS 2015)
  • Grab the instantsetup bundle of the latest MGOS release and spin it up
    • Grab the matching "buildpack" for this release (need to prepare this)
    • Grab and install SWIG 3.0.12
  • Do the main build/test
    • envsetup.cmd <major> <minor> <patch> <rev>
    • build.cmd
    • Build/run the .net test suite

TravisCI:

  • Use ubuntu14 x64 image if available
  • Use official ubuntu install script to headlessly install latest MGOS release and spin it up
    • Grab and extract headers from the "buildpack" for this release
    • Grab/build/install SWIG 3.0.12 (swigsetup.sh)
  • Do the main build/test
    • ./envsetup.sh
    • make on src/Bindings/DotNet
    • Build/run the .net test suite

Is there any new version of MapGuideDotNetApi ?

What Target CORE Framework will be compatible with MapGuideDotNetApi.3.1.1.9378-pre74.nupkg ?

I'm getting this error

dotnet add GoeMapping.CORE package MapGuideDotNetApi --version 3.1.1.9378-pre74

error: Package 'MapGuideDotNetApi' is incompatible with 'all' frameworks in project 'D:\Development\GOEMapping\Development\GoeMapping.CORE\GoeMapping.CORE\GoeMapping.CORE.csproj'.

Thanks for your help

Have IMake generate a MapGuideApi_Properties.i file instead of custom files

Previously, IMake would generate custom files for .net property implementations that the (super-modified) copy of SWIG would process to insert property definitions to .net proxy classes.

SWIG already has a mechanism for this

%attribute

So IMake should just generate a MapGuideApi_Properties.i file containing all the %attribute typemaps from the various property definitions scanned by IMake

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.