alliedmodders / ambuild Goto Github PK

AlliedModders C++ Build System

License: BSD 3-Clause "New" or "Revised" License

Python 99.24% Shell 0.12% C++ 0.18% HLSL 0.46%

ambuild's Introduction

AMBuild is a lightweight build system designed for performance and accuracy. It is geared toward C/C++ projects which require programmatic flexibility in their builds and precise control over C/C++ compiler flags.

AMBuild requires Python 3.3 or higher.

For more information, see: https://wiki.alliedmods.net/AMBuild

Installation

git clone https://github.com/alliedmodders/ambuild
pip install ./ambuild

AMBuild 2

AMBuild 2 is a highly efficient build system designed to replace "Alpha"-generation tools, such as SCons or Make. It is not a replacement for IDE project files, nor is it a front-end tool for generating other build system files, such as CMake. AMBuild is designed with three features in mind:

Accuracy. AMBuild guarantees that you never need to "clean" a build. Incremental builds should always produce the same exact result as a clean build; anything less is asking for trouble, and rebuilds are a waste of developer time.
Speed. Many build systems need to traverse the entire dependency graph. AMBuild only needs to find which files have changed. In addition, AMBuild will parallelize any independent tasks.
Programmability. Build scripts are written in Python, affording a great deal of flexibility for describing the build process.

Build scripts for AMBuild are parsed once upon configuration, and are responsible for defining tasks. If build scripts change, the build is automatically reconfigured. Out of box, build scripts support the following actions:

C/C++ compilation, linking, .rc compilation, and producing symbol files for symstore/breakpad.
File copying or symlinking for packaging.
Arbitrary shell commands.

AMBuild 1

AMBuild 1 was intended as a replacement for build systems such as SCons or Make. Its syntax is easier than Make and handles C/C++ dependencies automatically. Like most build systems, it performs a full recursive search for outdated files, which can make it slower for dependency graphs with many edges. It has no multiprocess support. Also unlike AMBuild 2, the dependency graph is not saved in between builds, which greatly reduces its incremental build accuracy and speed. C AMBuild 1 is installed alongside AMBuild 2 for backward compatibility, however it resides in an older namespace and has a completely separate API.

Contributing

AMBuild is written in Python.

Code is formatted using YAPF. If GitHub tells you there are style issues, you can use "yapf -r -i ." to fix them. You can get YAPF with pip ("pip install yapf").

Bugfixes are welcome, including to older API versions. New features are only added to the newest API.

AlliedModders developers can often be found in IRC (irc.gamesurge.net, #smdevs) if you have questions.

References

ambuild's People

Contributors

Stargazers

Watchers

ambuild's Issues

Can't compile with MSVC in normal command prompt

With latest AMBuild I've heard it should be able to detect MSVC in the normal command prompt, rather than needing to use the VS command prompt.

However, this doesn't seem to work for me:

On VS command prompt seems to work as expected:

Here's the output of vswhere -format json -products * -requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64 -utf8 if it helps:

vswhere output

[
  {
    "instanceId": "0559c9e4",
    "installDate": "2019-05-31T23:31:40Z",
    "installationName": "VisualStudio/16.7.5+30523.141",
    "installationPath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community",
    "installationVersion": "16.7.30523.141",
    "productId": "Microsoft.VisualStudio.Product.Community",
    "productPath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\Common7\\IDE\\devenv.exe",
    "state": 4294967295,
    "isComplete": true,
    "isLaunchable": true,
    "isPrerelease": false,
    "isRebootRequired": false,
    "displayName": "Visual Studio Community 2019",
    "description": "Powerful IDE, free for students, open-source contributors, and individuals",
    "channelId": "VisualStudio.16.Release",
    "channelUri": "https://aka.ms/vs/16/release/channel",
    "enginePath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\resources\\app\\ServiceHub\\Services\\Microsoft.VisualStudio.Setup.Service",
    "releaseNotes": "https://go.microsoft.com/fwlink/?LinkId=660893#16.7.5",
    "thirdPartyNotices": "https://go.microsoft.com/fwlink/?LinkId=660909",
    "updateDate": "2020-10-05T10:23:08.0919066Z",
    "catalog": {
      "buildBranch": "d16.7",
      "buildVersion": "16.7.30523.141",
      "id": "VisualStudio/16.7.5+30523.141",
      "localBuild": "build-lab",
      "manifestName": "VisualStudio",
      "manifestType": "installer",
      "productDisplayVersion": "16.7.5",
      "productLine": "Dev16",
      "productLineVersion": "2019",
      "productMilestone": "RTW",
      "productMilestoneIsPreRelease": "False",
      "productName": "Visual Studio",
      "productPatchVersion": "5",
      "productPreReleaseMilestoneSuffix": "1.0",
      "productSemanticVersion": "16.7.5+30523.141",
      "requiredEngineVersion": "2.7.3132.26759"
    },
    "properties": {
      "campaignId": "",
      "channelManifestId": "VisualStudio.16.Release/16.7.5+30523.141",
      "nickname": "",
      "setupEngineFilePath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\vs_installershell.exe"
    }
  },
  {
    "instanceId": "647b8f09",
    "installDate": "2019-10-12T03:46:44Z",
    "installationName": "VisualStudio/15.9.16+28307.858",
    "installationPath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Community",
    "installationVersion": "15.9.28307.858",
    "productId": "Microsoft.VisualStudio.Product.Community",
    "productPath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Community\\Common7\\IDE\\devenv.exe",
    "state": 4294967295,
    "isComplete": true,
    "isLaunchable": true,
    "isPrerelease": false,
    "isRebootRequired": false,
    "displayName": "Visual Studio Community 2017",
    "description": "Free, fully-featured IDE for students, open-source and individual developers",
    "channelId": "VisualStudio.15.Release",
    "channelUri": "https://aka.ms/vs/15/release/channel",
    "enginePath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\resources\\app\\ServiceHub\\Services\\Microsoft.VisualStudio.Setup.Service",
    "releaseNotes": "https://go.microsoft.com/fwlink/?LinkId=660692#15.9.16",
    "thirdPartyNotices": "https://go.microsoft.com/fwlink/?LinkId=660708",
    "updateDate": "2019-10-12T03:46:44.8291531Z",
    "catalog": {
      "buildBranch": "d15.9",
      "buildVersion": "15.9.28307.858",
      "id": "VisualStudio/15.9.16+28307.858",
      "localBuild": "build-lab",
      "manifestName": "VisualStudio",
      "manifestType": "installer",
      "productDisplayVersion": "15.9.16",
      "productLine": "Dev15",
      "productLineVersion": "2017",
      "productMilestone": "RTW",
      "productMilestoneIsPreRelease": "False",
      "productName": "Visual Studio",
      "productPatchVersion": "16",
      "productPreReleaseMilestoneSuffix": "1.0",
      "productRelease": "RTW",
      "productSemanticVersion": "15.9.16+28307.858",
      "requiredEngineVersion": "1.18.1049.33485"
    },
    "properties": {
      "campaignId": "",
      "channelManifestId": "VisualStudio.15.Release/15.9.16+28307.858",
      "nickname": "",
      "setupEngineFilePath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\vs_installershell.exe"
    }
  }
]

[clang/macOS/2.2 API] does not output build errors from individual source files

Using clang on macOS and the 2.2 API, if an individual source file fails to build there are no build errors indicating why it failed. This is caused by #123.

Commit a66fa6f:

[70439] clang -m64 -arch x86_64 -pipe -fno-strict-aliasing -Wall -Werror -Wno-unused -Wno-switch -Wno-array-bounds -msse -fvisibility=hidden -Wno-narrowing -Wno-sometimes-uninitialized -mmacosx-version-min=10.7 -fPIC -Wno-attributes -g3 -Dstricmp=strcasecmp -D_stricmp=strcasecmp -D_snprintf=snprintf -D_vsnprintf=vsnprintf -DHAVE_STDINT_H -DGNUC -DOSX -D_OSX -DPOSIX -DHAVE_CONFIG_H -I /Users/Tsunami/dev/sourcemod/public -I /Users/Tsunami/dev/sm-ripext/jansson/src -I /Users/Tsunami/dev/sm-ripext/jansson/include -I /Users/Tsunami/dev/sm-ripext/jansson/private_include -H -c /Users/Tsunami/dev/sm-ripext/jansson/src/hashtable.c -o hashtable.o
In file included from /Users/Tsunami/dev/sm-ripext/jansson/src/hashtable.c:34:
/Users/Tsunami/dev/sm-ripext/jansson/src/lookup3.h:52:11: fatal error: 'endian.h' file not found
# include <endian.h>    /* attempt to define endianness */
          ^~~~~~~~~~
1 error generated.
Build failed: libjansson/jansson/mac-x86_64/hashtable.o

Commit 6bd2f33:

[71229] clang -m64 -arch x86_64 -pipe -fno-strict-aliasing -Wall -Werror -Wno-unused -Wno-switch -Wno-array-bounds -msse -fvisibility=hidden -Wno-narrowing -Wno-sometimes-uninitialized -mmacosx-version-min=10.7 -fPIC -Wno-attributes -g3 -Dstricmp=strcasecmp -D_stricmp=strcasecmp -D_snprintf=snprintf -D_vsnprintf=vsnprintf -DHAVE_STDINT_H -DGNUC -DOSX -D_OSX -DPOSIX -DHAVE_CONFIG_H -I /Users/Tsunami/dev/sourcemod/public -I /Users/Tsunami/dev/sm-ripext/jansson/src -I /Users/Tsunami/dev/sm-ripext/jansson/include -I /Users/Tsunami/dev/sm-ripext/jansson/private_include -MD -MF hashtable.d -c /Users/Tsunami/dev/sm-ripext/jansson/src/hashtable.c -o hashtable.o
Traceback (most recent call last):
  File "/Users/Tsunami/Library/Python/3.8/lib/python/site-packages/ambuild2/task.py", line 85, in receive_task
    return self.process_task(channel, message)
  File "/Users/Tsunami/Library/Python/3.8/lib/python/site-packages/ambuild2/task.py", line 120, in process_task
    response = self.taskMap[task_type](message)
  File "/Users/Tsunami/Library/Python/3.8/lib/python/site-packages/ambuild2/task.py", line 70, in <lambda>
    'cxx': lambda message: self.doCompile(message),
  File "/Users/Tsunami/Library/Python/3.8/lib/python/site-packages/ambuild2/task.py", line 279, in doCompile
    out, err, paths = self.parseDependencies(tools_env, out, err, dep_type, dep_info)
  File "/Users/Tsunami/Library/Python/3.8/lib/python/site-packages/ambuild2/task.py", line 292, in parseDependencies
    with open(dep_info) as fp:
FileNotFoundError: [Errno 2] No such file or directory: 'hashtable.d'
Build failed: libjansson/jansson/mac-x86_64/hashtable.d

As you can see it only mentions that hashtable.d could not be found, not why hashtable.c failed to build.

Add copy-all-files-in-directory version of builder.AddCopy

Instead of having to manually list all files in a directory, allow to just use a wildcard to add all files. This would cleanup SourceMod's PackageScript.

https://github.com/alliedmodders/sourcemod/blob/e6129ab2d9bfadccd05567e4e542d1c397ba8cc5/tools/buildbot/PackageHelpers#L49-L62

Accented characters with VS2017 (French installation)

OS: Windows 10
Python : 2.7
IDE : Visual Studio Community 2017 (FR)

I got this issue when executing the command "ambuild" in /sourcemod/build/:

Traceback (most recent call last):
  File "D:\Developpements\Programmation\AlliedModders\python27\lib\site-packages\ambuild2\ipc\windows.py", line 405, in processMessage
    listener.receiveMessage(channel, message)
  File "D:\Developpements\Programmation\AlliedModders\python27\lib\site-packages\ambuild2\ipc\process.py", line 287, in receiveMessage
    self.listener.receiveMessage(self.child, message)
  File "D:\Developpements\Programmation\AlliedModders\python27\lib\site-packages\ambuild2\ipc\process.py", line 186, in receiveMessage
    return self.messageMap[message['id']](child, message)
  File "D:\Developpements\Programmation\AlliedModders\python27\lib\site-packages\ambuild2\task.py", line 436, in <lambda>
    'results': lambda child, message: self.processResults(message),
  File "D:\Developpements\Programmation\AlliedModders\python27\lib\site-packages\ambuild2\task.py", line 502, in processResults
    self.spewResult(message)
  File "D:\Developpements\Programmation\AlliedModders\python27\lib\site-packages\ambuild2\task.py", line 485, in spewResult
    util.WriteEncodedText(sys.stdout, message['stdout'])
  File "D:\Developpements\Programmation\AlliedModders\python27\lib\site-packages\ambuild2\util.py", line 452, in WriteEncodedText
    fd.write(text)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0' in position 144: ordinal not in range(128)
Received unexpected error from child process 5764: badmsg
Build failed.

I added "print" in util.py l.450 and I saw compiler warnings like:

d:\developpements\programmation\alliedmodders\sourcemod\extensions\curl\curl-src\lib\setup_once.h(373): warning C4005: 'EWOULDBLOCK' : redéfinition de macro
C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\ucrt\errno.h(125): note: voir la définition précédente de 'EWOULDBLOCK'

It looks like characters 'é', 'è'.. are not tolerated.

I reinstalled VS2017 in english and this issue doesn't appear anymore.

Standarize problem output

Would it be possible to implement a way to convert compiler specific problem output into a generic problem output?

This would allow for better integration into Visual Studio Code where you can define custom problem matchers, reference: Visual Studio Code: Tasks: Custom Problem Matcher

Console color doesn't reset properly on windows cmd/terminal if protoc is used

When protoc custom tools is used, and after the message found protoc XXXXXXXXXXXX is printed, the color is not reset to the default and is remained to be purple until changed by something else later.

-g3 switch clang

OS: Linux x64
Python: 3.6.6
CC: Clang 6.0.1

Every time I build SourcePawn using Clang I get -g3 switch specified. Doesn't matter if I pass --enable-optimize or not.

Passing --enable-debug causes that -g3 is specified twice because of https://github.com/alliedmodders/sourcepawn/blob/d653700024004082b2787b2d95e66eba7d2f598e/AMBuildScript#L77.

From what I found this behavior was added to ambuild back in 2013.
2f9a0be
https://bugs.alliedmods.net/show_bug.cgi?id=5997#c15

But then removed for GCC in 2015.
1864bb9#diff-1427703f7513811334a37d948e0648bcL101
1864bb9#diff-a119ca3177c6ded3d2f16a88cbe924d1R68

What is the reason of specifying -g3 switch by default?
Is there any way to disable that without manually tweaking ambuild sources?

PicklingError under python 3.9.13

I'm observing a pickling error during setup when run under python 3.9.13 (on Windows).

I saw there was a previously applied commit which fixed this for 3.10+, but I believe it may also need to be extended to earlier versions of python.

I'm not sure if it works on other versions of 3.9, but the easiest solution here is probably just to update the check to just skip for 3.9+ instead of 3.10+.

Specific error I've observed is as follows.

 _pickle.PicklingError: Can't pickle <function detect_distutils at 0x0000017FD6379160>: attribute lookup detect_distutils on __main__ failed

[Question] Could i upgrade AMBuild to use other compiler, e.g. Sourcepawn?

Hi. I have a repo with a lot of plugins and extensions.
Some plugins has own dir and own .inc files.

I want build whole tree and copy .smx / .so files to final package.

If i change .inc file i want rebuild all plugins who depends from that.

First of all this needed to test, run build for each commit.
Second is i want describe which plugins/exts i want to collect and make "distribution package", which i can deploy to the game server (e.g. by gitlab ci/cd)

I'm Python developer, and it's not big a problem make this, but it will be helpful if you give some advices. Thank you :)

Compiler has no attribute preprocessArgs

Hello,
I'm trying to configure AmBuild to perform a first build for AmxModX and this error is thrown:

Windows can't generate configs for ambuild.

Not sure did I missed some steps.
I was trying to build SourceMod with the Tutorial from Here.
But Windows can't generate configs for ambuild.

On Windows, ambuild or configure.py not working and thrown to this error.

D:\Code\Github\TestSourceMod\sourcemod\wbuild>python ..\configure.py --no-mysql --enable-optimize -s none --mms-path ../../mmsource-1.10/
Checking CC compiler (vendor test msvc)... ['C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.27.29110\\bin\\HostX64\\x86\\cl.exe', 'test.c', '-o', 'test-c.exe', '-nologo', '-showIncludes']
found msvc version 1927, x86
Checking CXX compiler (vendor test msvc)... ['C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.27.29110\\bin\\HostX64\\x86\\cl.exe', 'test.cpp', '-o', 'test-cxx.exe', '-nologo', '-showIncludes']
found msvc version 1927, x86
Warning: hl2sdk-episode1 was not found, and will not be included in build.
Warning: hl2sdk-ep2 was not found, and will not be included in build.
Warning: hl2sdk-css was not found, and will not be included in build.
Warning: hl2sdk-hl2dm was not found, and will not be included in build.
Warning: hl2sdk-dods was not found, and will not be included in build.
Warning: hl2sdk-sdk2013 was not found, and will not be included in build.
Warning: hl2sdk-tf2 was not found, and will not be included in build.
Warning: hl2sdk-l4d was not found, and will not be included in build.
Warning: hl2sdk-nucleardawn was not found, and will not be included in build.
Warning: hl2sdk-l4d2 was not found, and will not be included in build.
Warning: hl2sdk-darkm was not found, and will not be included in build.
Warning: hl2sdk-swarm was not found, and will not be included in build.
Warning: hl2sdk-bgt was not found, and will not be included in build.
Warning: hl2sdk-eye was not found, and will not be included in build.
Warning: hl2sdk-csgo was not found, and will not be included in build.
Warning: hl2sdk-blade was not found, and will not be included in build.
Warning: hl2sdk-insurgency was not found, and will not be included in build.
Warning: hl2sdk-contagion was not found, and will not be included in build.
Warning: hl2sdk-bms was not found, and will not be included in build.
Warning: hl2sdk-doi was not found, and will not be included in build.
Traceback (most recent call last):
  File "C:\Users\Vox Nein\AppData\Roaming\Python\Python38\site-packages\ambuild2\frontend\v2_2\prep.py", line 132, in Configure
    if not cm.generate(options.generator):
  File "C:\Users\Vox Nein\AppData\Roaming\Python\Python38\site-packages\ambuild2\frontend\context_manager.py", line 93, in generate
    self.parseBuildScripts()
  File "C:\Users\Vox Nein\AppData\Roaming\Python\Python38\site-packages\ambuild2\frontend\v2_2\context_manager.py", line 50, in parseBuildScripts
    self.execContext(cx)
  File "C:\Users\Vox Nein\AppData\Roaming\Python\Python38\site-packages\ambuild2\frontend\v2_2\context_manager.py", line 130, in execContext
    exec(code, scriptGlobals)
  File "D:\Code\Github\TestSourceMod\sourcemod\AMBuildScript", line 713, in <module>
    SP = builder.Build('sourcepawn/AMBuildScript', {
  File "C:\Users\Vox Nein\AppData\Roaming\Python\Python38\site-packages\ambuild2\frontend\v2_2\context.py", line 157, in Build
    return self.cm.runBuildScript(self, path, vars or {})
  File "C:\Users\Vox Nein\AppData\Roaming\Python\Python38\site-packages\ambuild2\frontend\v2_2\context_manager.py", line 68, in runBuildScript
    return self.runBuildScriptImpl(context, path, vars or {})
  File "C:\Users\Vox Nein\AppData\Roaming\Python\Python38\site-packages\ambuild2\frontend\v2_2\context_manager.py", line 119, in runBuildScriptImpl
    scriptGlobals = self.execContext(cx)
  File "C:\Users\Vox Nein\AppData\Roaming\Python\Python38\site-packages\ambuild2\frontend\v2_2\context_manager.py", line 130, in execContext
    exec(code, scriptGlobals)
  File "D:\Code\Github\TestSourceMod\sourcemod\sourcepawn\AMBuildScript", line 351, in <module>
    sp.BuildSpcomp()
  File "D:\Code\Github\TestSourceMod\sourcemod\sourcepawn\AMBuildScript", line 268, in BuildSpcomp
    spcomp = self.BuildForArch('compiler/AMBuilder', cxx)
  File "D:\Code\Github\TestSourceMod\sourcemod\sourcepawn\AMBuildScript", line 331, in BuildForArch
    rvalue = builder.Build(scripts, new_vars)
  File "C:\Users\Vox Nein\AppData\Roaming\Python\Python38\site-packages\ambuild2\frontend\v2_2\context.py", line 157, in Build
    return self.cm.runBuildScript(self, path, vars or {})
  File "C:\Users\Vox Nein\AppData\Roaming\Python\Python38\site-packages\ambuild2\frontend\v2_2\context_manager.py", line 68, in runBuildScript
    return self.runBuildScriptImpl(context, path, vars or {})
  File "C:\Users\Vox Nein\AppData\Roaming\Python\Python38\site-packages\ambuild2\frontend\v2_2\context_manager.py", line 119, in runBuildScriptImpl
    scriptGlobals = self.execContext(cx)
  File "C:\Users\Vox Nein\AppData\Roaming\Python\Python38\site-packages\ambuild2\frontend\v2_2\context_manager.py", line 130, in execContext
    exec(code, scriptGlobals)
  File "D:\Code\Github\TestSourceMod\sourcemod\sourcepawn\compiler\AMBuilder", line 80, in <module>
    rvalue = builder.Add(binary)
  File "C:\Users\Vox Nein\AppData\Roaming\Python\Python38\site-packages\ambuild2\frontend\v2_2\context.py", line 212, in Add
    taskbuilder.finish(self)
  File "C:\Users\Vox Nein\AppData\Roaming\Python\Python38\site-packages\ambuild2\frontend\v2_2\cpp\builders.py", line 559, in finish
    self.buildModules(cx)
  File "C:\Users\Vox Nein\AppData\Roaming\Python\Python38\site-packages\ambuild2\frontend\v2_2\cpp\builders.py", line 467, in buildModules
    self.buildModule(cx, module)
  File "C:\Users\Vox Nein\AppData\Roaming\Python\Python38\site-packages\ambuild2\frontend\v2_2\cpp\builders.py", line 529, in buildModule
    obj_item = builder.buildItem(inputObj, sourceName, sourceFile)
  File "C:\Users\Vox Nein\AppData\Roaming\Python\Python38\site-packages\ambuild2\frontend\v2_2\cpp\builders.py", line 215, in buildItem
    return self.buildRcItem(inputObj, sourceFile, encodedName)
  File "C:\Users\Vox Nein\AppData\Roaming\Python\Python38\site-packages\ambuild2\frontend\v2_2\cpp\builders.py", line 259, in buildRcItem
    assert not self.vendor.emits_dependency_files
AttributeError: 'MSVC' object has no attribute 'emits_dependency_files'
Configure failed: 'MSVC' object has no attribute 'emits_dependency_files'

On Windows WSL Ubuntu(Same machine, Same Code Path), it works and build successfully.

xein@Xein:/mnt/d/Code/Github/testsourcemod/sourcemod/build$ sudo python3 ../configure.py --no-mysql --enable-optimize -s none --mms-path ../../mmsource-1.10/
[sudo] password for xein:
Checking CC compiler (vendor test gcc)... ['clang', '-m32', 'test.c', '-o', 'test-c']
found clang version 10.0, x86
Checking CXX compiler (vendor test gcc)... ['clang++', '-m32', 'test.cpp', '-o', 'test-cxx']
found clang version 10.0, x86
Warning: hl2sdk-episode1 was not found, and will not be included in build.
Warning: hl2sdk-ep2 was not found, and will not be included in build.
Warning: hl2sdk-css was not found, and will not be included in build.
Warning: hl2sdk-hl2dm was not found, and will not be included in build.
Warning: hl2sdk-dods was not found, and will not be included in build.
Warning: hl2sdk-sdk2013 was not found, and will not be included in build.
Warning: hl2sdk-tf2 was not found, and will not be included in build.
Warning: hl2sdk-l4d was not found, and will not be included in build.
Warning: hl2sdk-nucleardawn was not found, and will not be included in build.
Warning: hl2sdk-l4d2 was not found, and will not be included in build.
Warning: hl2sdk-csgo was not found, and will not be included in build.
Warning: hl2sdk-insurgency was not found, and will not be included in build.
Warning: hl2sdk-bms was not found, and will not be included in build.
Warning: hl2sdk-doi was not found, and will not be included in build.
xein@Xein:/mnt/d/Code/Github/testsourcemod/sourcemod/build$

So, I think there is some bugs on building with MSVC?

AMBuild Windows Error

@dvander I have an error for you from GitHub actions, environment is: Windows x64 Python 3.6

2020-08-14T15:44:42.7861401Z Traceback (most recent call last):
2020-08-14T15:44:42.7861665Z Checking CC compiler (vendor test msvc)... ['cl', 'test.c', '-o', 'test.exe', '-nologo', '-showIncludes']
2020-08-14T15:44:42.7863046Z   File "C:\hostedtoolcache\windows\Python\3.6.8\x64\lib\site-packages\ambuild2\frontend\v2_1\prep.py", line 160, in Configure
2020-08-14T15:44:42.7863226Z found msvc version 1926
2020-08-14T15:44:42.7863673Z     if not cm.generate(options.generator):
2020-08-14T15:44:42.7863838Z Checking CXX compiler (vendor test msvc)... ['cl', 'test.cpp', '-o', 'testp.exe', '-nologo', '-showIncludes']
2020-08-14T15:44:42.7864386Z   File "C:\hostedtoolcache\windows\Python\3.6.8\x64\lib\site-packages\ambuild2\frontend\context_manager.py", line 93, in generate
2020-08-14T15:44:42.7864544Z found msvc version 1926
2020-08-14T15:44:42.7864873Z     self.parseBuildScripts()
2020-08-14T15:44:42.7865212Z   File "C:\hostedtoolcache\windows\Python\3.6.8\x64\lib\site-packages\ambuild2\frontend\v2_1\context_manager.py", line 57, in parseBuildScripts
2020-08-14T15:44:42.7865493Z     self.execContext(cx)
2020-08-14T15:44:42.7865914Z   File "C:\hostedtoolcache\windows\Python\3.6.8\x64\lib\site-packages\ambuild2\frontend\v2_1\context_manager.py", line 137, in execContext
2020-08-14T15:44:42.7866193Z     exec(code, scriptGlobals)
2020-08-14T15:44:42.7866481Z   File "D:\a\ambuild\ambuild\tests\modules\AMBuildScript", line 2, in <module>
2020-08-14T15:44:42.7866739Z     builder.DetectCxx()
2020-08-14T15:44:42.7867047Z   File "C:\hostedtoolcache\windows\Python\3.6.8\x64\lib\site-packages\ambuild2\frontend\v2_1\context.py", line 135, in DetectCxx
2020-08-14T15:44:42.7867337Z     self.cxx_ = self.generator_.detectCompilers(options).clone()
2020-08-14T15:44:42.7867661Z   File "C:\hostedtoolcache\windows\Python\3.6.8\x64\lib\site-packages\ambuild2\frontend\v2_1\amb2_gen.py", line 34, in detectCompilers
2020-08-14T15:44:42.7867943Z     self.compiler = self.base_compiler.clone()
2020-08-14T15:44:42.7868186Z AttributeError: 'tuple' object has no attribute 'clone'
2020-08-14T15:44:42.7868431Z Configure failed: 'tuple' object has no attribute 'clone'
2020-08-14T15:44:42.9490130Z Traceback (most recent call last):
2020-08-14T15:44:42.9490988Z   File "c:\hostedtoolcache\windows\python\3.6.8\x64\lib\runpy.py", line 193, in _run_module_as_main
2020-08-14T15:44:42.9491484Z     "__main__", mod_spec)
2020-08-14T15:44:42.9491978Z   File "c:\hostedtoolcache\windows\python\3.6.8\x64\lib\runpy.py", line 85, in _run_code
2020-08-14T15:44:42.9492507Z     exec(code, run_globals)
2020-08-14T15:44:42.9492816Z   File "C:\hostedtoolcache\windows\Python\3.6.8\x64\Scripts\ambuild.exe\__main__.py", line 7, in <module>
2020-08-14T15:44:42.9493151Z   File "c:\hostedtoolcache\windows\python\3.6.8\x64\lib\site-packages\ambuild2\run.py", line 181, in cli_run
2020-08-14T15:44:42.9493428Z     if not Build(os.path.abspath(folder), options, argv):
2020-08-14T15:44:42.9493729Z   File "c:\hostedtoolcache\windows\python\3.6.8\x64\lib\site-packages\ambuild2\run.py", line 131, in Build
2020-08-14T15:44:42.9494000Z     with Context(buildPath, options, argv) as cx:
2020-08-14T15:44:42.9494471Z   File "c:\hostedtoolcache\windows\python\3.6.8\x64\lib\site-packages\ambuild2\context.py", line 40, in __init__
2020-08-14T15:44:42.9494755Z     with open(os.path.join(self.cacheFolder, 'vars'), 'rb') as fp:
2020-08-14T15:44:42.9495018Z FileNotFoundError: [Errno 2] No such file or directory: 'D:\\a\\ambuild\\ambuild\\objdir\\.ambuild2\\vars'

Add support for msvc17 "2022" gen

VS 2022 released recently. Updating gen.py YearMap with '2022': 17, didnt appear to be enough, as VS asked upon launching to upgrade the microsoft toolset

Can't generate VS projects for static library and executable with the same name

When there is a StaticLibrary and a Program with the same name in a builder instance VS project file generation fails.

C:\Users\User\Documents\GitHub\sourcepawn\vs>python ..\configure.py --enable-debug -g vs --build=all
Path exp\compiler\spcomp - Default\spcomp.pdb already exists as: output
Traceback (most recent call last):
  File "..\configure.py", line 41, in <module>
    run.Configure()
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\site-packages\ambuild2\frontend\v2_0\prep.py", line 121, in Configure
    if not builder.generate():
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\site-packages\ambuild2\frontend\v2_0\base\gen.py", line 274, in generate
    self.parseBuildScripts()
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\site-packages\ambuild2\frontend\v2_0\base\gen.py", line 183, in parseBuildScripts
    self.evalScript(root)
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\site-packages\ambuild2\frontend\v2_0\base\gen.py", line 243, in evalScript
    exec(code, new_vars)
  File "C:\Users\User\Documents\GitHub\sourcepawn\AMBuildScript", line 267, in <module>
    sp.BuildSuite()
  File "C:\Users\User\Documents\GitHub\sourcepawn\AMBuildScript", line 251, in BuildSuite
    self.BuildExperimental()
  File "C:\Users\User\Documents\GitHub\sourcepawn\AMBuildScript", line 243, in BuildExperimental
    builder.RunBuildScripts(self.exp_scripts, self.vars)
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\site-packages\ambuild2\frontend\v2_0\base\gen.py", line 117, in RunBuildScripts
    self.generator.evalScript(script, vars)
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\site-packages\ambuild2\frontend\v2_0\base\gen.py", line 243, in evalScript
    exec(code, new_vars)
  File "C:\Users\User\Documents\GitHub\sourcepawn\exp\compiler\AMBuilder", line 83, in <module>
    builder.Add(binary)
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\site-packages\ambuild2\frontend\v2_0\base\gen.py", line 121, in Add
    return taskbuilder.generate(self.generator, self)
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\site-packages\ambuild2\frontend\v2_0\vs\cxx.py", line 56, in <lambda>
    builder.generate = lambda generator, cx: self.generate(generator, cx)[0]
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\site-packages\ambuild2\frontend\v2_0\vs\cxx.py", line 65, in generate
    return self.generate_combined(generator, cx)
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\site-packages\ambuild2\frontend\v2_0\vs\cxx.py", line 85, in generate_combined
    pdbNode = generator.addOutput(cx, pdbFile, node)
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\site-packages\ambuild2\frontend\v2_0\vs\gen.py", line 161, in addOutput
    self.ensureUnique(path)
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\site-packages\ambuild2\frontend\v2_0\vs\gen.py", line 118, in ensureUnique
    raise Exception('Path {0} already exists as: {1}'.format(path, entry.kind))
Exception: Path exp\compiler\spcomp - Default\spcomp.pdb already exists as: output

alliedmodders/sourcepawn#27

pip future deprecation: inplace built

Just checkout, needed tests and confirmation that everything ready.

  DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.
   pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555.

[windows] how to get lib name

It seem that couldn't get lib name. For example, a Library(Not StaticLibrary), I want to get the .lib name, but I can't get it, only two attribute "binary"(.exe, *.lib, *.dll) and "debug"(debug symbol)

Generated Visual Studio projects generating files in wrong output folder

ambuild creates folders for all configurations of the generated VS projects. They are prefixed with the project name like sourcemod.logic - Default.
Building the sourcemod.logic.vcxproj will put the files in a folder named just Default and the sourcemod.logic - Default folder put in place by ambuild so carefully is left empty 😢 Building another project like sourcemod_mm will output into the same Default folder instead of the project specific one.

Furthermore the projects containing more than one "Default" configuration like sourcemod and sdktools having one for each engine branch, are getting folders created which are missing the project name as prefix.
So there's just a folder called Debug - csgo instead of multiple ones per project like sourcemod - Debug - csgo and sdktools - Debug - csgo. Building the projects actually create the files in the Debug - csgo folder, instead of a project specific one.

Switch setup.py to use setuptools

Discussion is currently going on at bpo-41282 to depreciate and remove distutils, AMBuild uses distutils for installation, but it doesn't support egg info generation which is required to uninstall AMBuild with pip.

Note: pip can install source distributions from an AMBuild repo clone/zip extract.
Note 2: setuptools mimics distutils to the point where minimal work will be required for AMBuild to switch.

Configure script fails if compiler is in PATH on Windows

When the configure script is launched, it runs through find_default_compiler since cl.exe can be found in PATH. However, the subprocess launched through subprocess.Popen do not have environment variables set or is not able to use PATH, so the function needs to have the environment variables passed to it somehow.

how to enable exceptions?

Minor: Add 'zip_safe=False' to setup.py 'setup' arguments.

While reviewing the build log for alliedmodders/sourcemod#1313, I noticed that python complained about the lack of 'zip_safe' argument:

python.exe : zip_safe flag not set; analyzing archive contents...
At C:\projects\sourcemod\tools\checkout-deps.ps1:96 char:1
+ & python setup.py install
+ ~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (zip_safe flag n...ive contents...:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError
 
ambuild2.frontend.v2_1.tools.fxc: module references __file__

Admittedly, I didn't review the source code of AMBuild completely, nor did I not add the argument and set it to False in #69
Reviewing the file suggests AMBuild2 is not really safe for being executed from a zip file.

Fixing this is as simple as adding 'zip_safe=False' into the method arguments for "setup()" inside 'setup.py'

AMBuild V2.2 idea: Proper package versioning.

Currently AMBuild has a problem with versioning itself with the installer, setuptools and by extension pip sees it as version 2.0.

For 2.2, we should start better marking the version so upgrades won't require the "--force" argument.

There is an extension called setuptools-git-versioning that uses git describe to obtain version info. This will also allow users to pin an AMBuild version to a specific API version.

Also: @dvander can you create an API 2.2 milestone?

Visual Studio project generation broken after Dep removal

Trying to generate project files for SourceMod fails due to a missing import which was removed in #119

``´
python ..\configure.py --no-mysql --enable-debug -s css,csgo -g vs
Traceback (most recent call last):
File "C:\Users\Jannik\AppData\Local\Programs\Python\Python38\lib\site-packages\ambuild2\frontend\v2_2\prep.py", line 132, in Configure
if not cm.generate(options.generator):
File "C:\Users\Jannik\AppData\Local\Programs\Python\Python38\lib\site-packages\ambuild2\frontend\context_manager.py", line 91, in generate
self.createGenerator(name)
File "C:\Userspython ..\configure.py --no-mysql --enable-debug -s css,csgo -g vs
Traceback (most recent call last):
File "C:\Python38\lib\site-packages\ambuild2\frontend\v2_2\prep.py", line 132, in Configure
if not cm.generate(options.generator):
File "C:\Python38\lib\site-packages\ambuild2\frontend\context_manager.py", line 91, in generate
self.createGenerator(name)
File "C:\Python38\lib\site-packages\ambuild2\frontend\v2_2\context_manager.py", line 177, in createGenerator
from ambuild2.frontend.v2_2.vs.gen import Generator
File "C:\Python38\lib\site-packages\ambuild2\frontend\v2_2\vs\gen.py", line 18, in
from ambuild2.frontend.v2_2.vs import cxx
File "C:\Python38\lib\site-packages\ambuild2\frontend\v2_2\vs\cxx.py", line 24, in
from ambuild2.frontend.v2_2.vs import export_vcxproj
File "C:\Python38\lib\site-packages\ambuild2\frontend\v2_2\vs\export_vcxproj.py", line 21, in
from ambuild2.frontend.v2_2.cpp.builders import Dep
ImportError: cannot import name 'Dep' from 'ambuild2.frontend.v2_2.cpp.builders' (C:\Python38\lib\site-packages\ambuild2\frontend\v2_2\cpp\builders.py)
Configure failed: cannot import name 'Dep' from 'ambuild2.frontend.v2_2.cpp.builders' (C:\Python38\lib\site-packages\ambuild2\frontend\v2_2\cpp\builders.py)

[Suggestion] Generate compilation database

Some IDEs cannot interact with the source code until the config of a (known) builder is loaded. One such IDE is Clion.
My suggestion is to add ability to generate a compilation database.

https://intellij-support.jetbrains.com/hc/en-us/community/posts/4403148099218/comments/4404294055954
https://clang.llvm.org/docs/JSONCompilationDatabase.html

TypeError while configuring sourcemod with python 3

I tried to use python 3 for ambuild, but i've run into an error which, i think, is a bug.
I used a fresh virtual environment for ambuild and the latest sources from github.
The following is the output, which hopefully contains all information you need.

python3 ../sourcemod/configure.py
Checking CC compiler (vendor test gcc)... ['cc', 'test.c', '-o', 'test']
found gcc version 4.7
Checking CXX compiler (vendor test gcc)... ['c++', 'test.cpp', '-o', 'testp']
found gcc version 4.7
Traceback (most recent call last):
  File "../sourcemod/configure.py", line 34, in <module>
    run.Configure()
  File "/home/impact/Sourcemod/sourcemod-virtualenv/lib/python3.2/site-packages/ambuild2/frontend/prep.py", line 121, in Configure
    if not builder.generate():
  File "/home/impact/Sourcemod/sourcemod-virtualenv/lib/python3.2/site-packages/ambuild2/frontend/base_gen.py", line 235, in generate
    self.parseBuildScripts()
  File "/home/impact/Sourcemod/sourcemod-virtualenv/lib/python3.2/site-packages/ambuild2/frontend/base_gen.py", line 165, in parseBuildScripts
    self.evalScript(root)
  File "/home/impact/Sourcemod/sourcemod-virtualenv/lib/python3.2/site-packages/ambuild2/frontend/base_gen.py", line 204, in evalScript
    exec(code, new_vars)
  File "/home/impact/Sourcemod/sourcemod/AMBuildScript", line 469, in <module>
    SM.configure()
  File "/home/impact/Sourcemod/sourcemod/AMBuildScript", line 177, in configure
    if have_clang or (have_gcc and cxx.version >= '4'):
TypeError: unorderable types: Version() >= str()

If you need more information, just tell me and i provide them.

how to make a dll without any source file only .rc file

As the title, when I create a build script for a project that only has a resource file, and my script is

binary = cfg.Library(builder, 'project_name', builder.target.arch)
compiler = binary.compiler
compiler.linkflags += [
    '/NOENTRY',
]
compiler.includes += [
    os.path.join(builder.currentSourcePath),
]

binary.sources += [
    'project_resource.rc',
]

builder.Add(binary)

and it will tell me the error, but the dll file will be generated, the error message will like this :
Expected output file, but not found: project.exp

how to do it correctly

Custom tools are ignored in project generators

Currently, output of custom tools like protoc, isn't included in the generated project files that's done via --gen option. Since in this particular example with protoc, the output files are placed in the build obj location and are used by the ambuild process, but are never evaluated for the project generation, which causes projects solutions to be unable to see and work with anything that depends on the generated content without manual changes.

Steps to reproduce would be to generate a VS project solution for metamod-source master and attempt to compile it within the MSVC.

AMBuild is currently broken

#89 has broke AMBuild as evident in AppVeyor and Travis-CI

Traceback (From AppVeyor):

Traceback (most recent call last):
  File "../configure.py", line 42, in <module>
    parser.Configure()
  File "c:\python27\lib\site-packages\ambuild2\frontend\v2_1\prep.py", line 154, in Configure
    from ambuild2.frontend.v2_1.context_manager import ContextManager
  File "c:\python27\lib\site-packages\ambuild2\frontend\v2_1\context_manager.py", line 28, in <module>
    from ambuild2.frontend.v2_1.cpp import detect
  File "c:\python27\lib\site-packages\ambuild2\frontend\v2_1\cpp\__init__.py", line 1, in <module>
    from ambuild2.frontend.v2_1.cpp.compiler import Compiler
  File "c:\python27\lib\site-packages\ambuild2\frontend\v2_1\cpp\compiler.py", line 20, in <module>
    from ambuild2.frontend.v2_1.cpp import builders
  File "c:\python27\lib\site-packages\ambuild2\frontend\v2_1\cpp\builders.py", line 21, in <module>
    from ambuild2.frontend.v2_1 import cpp
ImportError: cannot import name cpp
Command exited with code 1

FileNotFoundError on windows

I run into this a lot, but it seems only now it's interrupting my workflow. Tried a re-install of ambuild, and I'm on the latest version. Here's the stack trace I get from the console output when trying to build sourcemod on msvc 1900

Spawned taskmaster (pid: 5832)
Spawned worker (pid: 7424)
Spawned worker (pid: 10856)
Spawned worker (pid: 16956)
Spawned worker (pid: 16856)
Spawned worker (pid: 8820)
Spawned worker (pid: 11536)
Spawned worker (pid: 14548)
Spawned worker (pid: 10864)
Spawned worker (pid: 9268)
Spawned worker (pid: 11800)
Spawned worker (pid: 12780)
Spawned worker (pid: 1220)
Spawned worker (pid: 1904)
Spawned worker (pid: 8284)
Spawned worker (pid: 10020)
Spawned worker (pid: 5168)
Spawned worker (pid: 3356)
Spawned worker (pid: 12752)
Spawned worker (pid: 3996)
Spawned worker (pid: 16520)
Traceback (most recent call last):
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\ipc\windows.py", line 405, in processMessage
    listener.receiveMessage(channel, message)
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\ipc\impl.py", line 67, in receiveMessage
    self.listener.receiveMessage(channel, message)
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\ipc\process.py", line 150, in receiveMessage
    return self.messageMap[message['id']](channel, message)
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\task.py", line 53, in <lambda>
    'task': lambda channel, message: self.receiveTask(channel, message)
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\task.py", line 93, in receiveTask
    response = self.taskMap[task_type](message)
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\task.py", line 60, in <lambda>
    'rc': lambda message: self.doResource(message),
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\task.py", line 236, in doResource
    p, out, err = util.Execute(rc_argv)
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\util.py", line 164, in Execute
    shell=shell
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\subprocess.py", line 707, in __init__
    restore_signals, start_new_session)
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\subprocess.py", line 990, in _execute_child
    startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
[8284] Child process terminating abnormally.
Traceback (most recent call last):
Traceback (most recent call last):
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\ipc\windows.py", line 405, in processMessage
    listener.receiveMessage(channel, message)
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\ipc\impl.py", line 67, in receiveMessage
    self.listener.receiveMessage(channel, message)
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\ipc\process.py", line 150, in receiveMessage
    return self.messageMap[message['id']](channel, message)
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\task.py", line 53, in <lambda>
    'task': lambda channel, message: self.receiveTask(channel, message)
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\task.py", line 93, in receiveTask
    response = self.taskMap[task_type](message)
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\task.py", line 60, in <lambda>
    'rc': lambda message: self.doResource(message),
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\task.py", line 236, in doResource
    p, out, err = util.Execute(rc_argv)
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\util.py", line 164, in Execute
    shell=shell
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\subprocess.py", line 707, in __init__
    restore_signals, start_new_session)
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\subprocess.py", line 990, in _execute_child
    startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
Traceback (most recent call last):
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\ipc\windows.py", line 405, in processMessage
    listener.receiveMessage(channel, message)
[7424] Child process terminating abnormally.
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\ipc\windows.py", line 405, in processMessage
    listener.receiveMessage(channel, message)
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\ipc\impl.py", line 67, in receiveMessage
    self.listener.receiveMessage(channel, message)
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\ipc\process.py", line 150, in receiveMessage
    return self.messageMap[message['id']](channel, message)
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\task.py", line 53, in <lambda>
    'task': lambda channel, message: self.receiveTask(channel, message)
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\task.py", line 93, in receiveTask
    response = self.taskMap[task_type](message)
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\task.py", line 60, in <lambda>
    'rc': lambda message: self.doResource(message),
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\task.py", line 236, in doResource
    p, out, err = util.Execute(rc_argv)
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\util.py", line 164, in Execute
    shell=shell
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\subprocess.py", line 707, in __init__
    restore_signals, start_new_session)
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\subprocess.py", line 990, in _execute_child
    startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
[10856] Child process terminating abnormally.
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\ipc\impl.py", line 67, in receiveMessage
    self.listener.receiveMessage(channel, message)
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\ipc\process.py", line 150, in receiveMessage
    return self.messageMap[message['id']](channel, message)
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\task.py", line 53, in <lambda>
    'task': lambda channel, message: self.receiveTask(channel, message)
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\task.py", line 93, in receiveTask
    response = self.taskMap[task_type](message)
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\task.py", line 60, in <lambda>
    'rc': lambda message: self.doResource(message),
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\task.py", line 236, in doResource
    p, out, err = util.Execute(rc_argv)
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\site-packages\ambuild2\util.py", line 164, in Execute
    shell=shell
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\subprocess.py", line 707, in __init__
    restore_signals, start_new_session)
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python36-32\lib\subprocess.py", line 990, in _execute_child
    startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
[12752] Child process terminating abnormally.
Crashed trying to perform update:

test-c.exe and test-cxx.exe detected as trojans

While trying to run configure.py Kaspersky flagged test-c.exe as VHO:Trojan.Win32.Convagent.gen

https://www.virustotal.com/gui/file/7d765ec6f4771f4427bc05cea58b7058908c699735666ab245b5194e7d3d7bb9?nocache=1
https://www.virustotal.com/gui/file/684321975e7a5498fb45f635ac3ad68ac8fd3f1e4da43defb7228e7aaeac9b47?nocache=1

Drop Python 2 support

At this point in time, Python 2's life is very much over in favour of Python 3, AMBuild should be upgraded to drop Python 2 support some time soon, allowing for better performance overall.

Should Python 2 support still be required further though, a stop-gap solution will be relying on the 'six' module, as it's a Python 2/3 support library. Ultimately though, Python 2 should be dropped.

On the flip side, Python 3 introduced new libraries that AMBuild can take full advantage of, one of them is name 'pathlib', a platform independent-ish package for handling path/pathlike objects. A new interface was also introduced called 'os.PathLike' and most Python methods now accept string/PathLike objects.

error when installing on Windows 11

PS C:\Users\dev\Downloads> pip install .\ambuild\
Processing c:\users\dev\downloads\ambuild
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [42 lines of output]
      Traceback (most recent call last):
        File "C:\Users\dev\AppData\Local\Programs\Python\Python312\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
          main()
        File "C:\Users\dev\AppData\Local\Programs\Python\Python312\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\dev\AppData\Local\Programs\Python\Python312\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\dev\AppData\Local\Temp\pip-build-env-0irrx5re\overlay\Lib\site-packages\setuptools\build_meta.py", line 355, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\dev\AppData\Local\Temp\pip-build-env-0irrx5re\overlay\Lib\site-packages\setuptools\build_meta.py", line 325, in _get_build_requires
          self.run_setup()
        File "C:\Users\dev\AppData\Local\Temp\pip-build-env-0irrx5re\overlay\Lib\site-packages\setuptools\build_meta.py", line 507, in run_setup
          super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script)
        File "C:\Users\dev\AppData\Local\Temp\pip-build-env-0irrx5re\overlay\Lib\site-packages\setuptools\build_meta.py", line 341, in run_setup
          exec(code, locals())
        File "<string>", line 26, in <module>
        File "C:\Users\dev\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\process.py", line 121, in start
          self._popen = self._Popen(self)
                        ^^^^^^^^^^^^^^^^^
        File "C:\Users\dev\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\context.py", line 224, in _Popen
          return _default_context.get_context().Process._Popen(process_obj)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\dev\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\context.py", line 337, in _Popen
          return Popen(process_obj)
                 ^^^^^^^^^^^^^^^^^^
        File "C:\Users\dev\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\popen_spawn_win32.py", line 94, in __init__
          reduction.dump(process_obj, to_child)
        File "C:\Users\dev\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\reduction.py", line 60, in dump
          ForkingPickler(file, protocol).dump(obj)
      _pickle.PicklingError: Can't pickle <function detect_distutils at 0x0000016E6E732F20>: attribute lookup detect_distutils on __main__ failed
      Traceback (most recent call last):
        File "<string>", line 1, in <module>
        File "C:\Users\dev\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\spawn.py", line 113, in spawn_main
          new_handle = reduction.duplicate(pipe_handle,
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\dev\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\reduction.py", line 79, in duplicate
          return _winapi.DuplicateHandle(
                 ^^^^^^^^^^^^^^^^^^^^^^^^
      PermissionError: [WinError 5] Access is denied
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Tried in user and admin powershell

Can't specify path in hl2sdk-manifests option

While specifying an absolute path outside of the current working directory, the call to computeScriptPaths in context_manager.py,
will badly determine the location of the script and therefore concatenate an absolute path with another absolute path, leading to an error.

HL2SDKCS2=/very/long/path/on/hdd/data/Git/hl2sdk python ../configure.py -s cs2 --hl2sdk-manifests /very/long/path/on/hdd/data/Git/metamod-source/hl2sdk-manifests/
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.10/site-packages/ambuild2/frontend/v2_2/prep.py", line 156, in Configure
    if not cm.generate(options.generator):
  File "/home/user/.local/lib/python3.10/site-packages/ambuild2/frontend/context_manager.py", line 93, in generate
    self.parseBuildScripts()
  File "/home/user/.local/lib/python3.10/site-packages/ambuild2/frontend/v2_2/context_manager.py", line 50, in parseBuildScripts
    self.execContext(cx)
  File "/home/user/.local/lib/python3.10/site-packages/ambuild2/frontend/v2_2/context_manager.py", line 152, in execContext
    exec(code, scriptGlobals)
  File "/very/long/path/on/hdd/data/Git/metamod-source/samples/s2_sample_mm/AMBuildScript", line 68, in <module>
    SdkHelpers = builder.Eval(os.path.join(hl2sdk_manifests, 'SdkHelpers.ambuild'), {
  File "/home/user/.local/lib/python3.10/site-packages/ambuild2/frontend/v2_2/context.py", line 115, in Eval
    return self.cm.evalScript(self, path, vars or {})
  File "/home/user/.local/lib/python3.10/site-packages/ambuild2/frontend/v2_2/context_manager.py", line 60, in evalScript
    obj = self.importScriptImpl(context, path, vars or {})
  File "/home/user/.local/lib/python3.10/site-packages/ambuild2/frontend/v2_2/context_manager.py", line 81, in importScriptImpl
    self.generator.addConfigureFile(parent, scriptPath)
  File "/home/user/.local/lib/python3.10/site-packages/ambuild2/frontend/amb2_gen.py", line 744, in addConfigureFile
    self.db.add_or_update_script(path)
  File "/home/user/.local/lib/python3.10/site-packages/ambuild2/database.py", line 826, in add_or_update_script
    stamp = os.path.getmtime(path)
  File "/usr/lib/python3.10/genericpath.py", line 55, in getmtime
    return os.stat(filename).st_mtime
FileNotFoundError: [Errno 2] No such file or directory: '/very/long/path/on/hdd/data/Git/metamod-source/samples/s2_sample_mm/very/long/path/on/hdd/data/Git/metamod-source/hl2sdk-manifests/SdkHelpers.ambuild'

Furthermore it seems that this is also the case when trying to specify a relative path:

HL2SDKCS2=/very/long/path/on/hdd/data/Git/hl2sdk python ../configure.py -s cs2 --hl2sdk-manifests ../../hl2sdk-manifests/
Script File: ../../hl2sdk-manifests/SdkHelpers.ambuild
Source Path: /very/long/path/on/hdd/data/Git/metamod-source/samples/s2_sample_mm
Both: /very/long/path/on/hdd/data/Git/metamod-source/samples/s2_sample_mm/../../hl2sdk-manifests/SdkHelpers.ambuild
Checking CC compiler (vendor test gcc)... ['clang', '-m32', 'test.c', '-o', 'test-c']
found clang version 14.0, x86
Checking CXX compiler (vendor test gcc)... ['clang++', '-m32', 'test.cpp', '-o', 'test-cxx']
found clang version 14.0, x86
Checking CC compiler (vendor test gcc)... ['clang', '-m64', 'test.c', '-o', 'test-c']
found clang version 14.0, x86_64
Checking CXX compiler (vendor test gcc)... ['clang++', '-m64', 'test.cpp', '-o', 'test-cxx']
found clang version 14.0, x86_64
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.10/site-packages/ambuild2/frontend/v2_2/prep.py", line 156, in Configure
    if not cm.generate(options.generator):
  File "/home/user/.local/lib/python3.10/site-packages/ambuild2/frontend/context_manager.py", line 93, in generate
    self.parseBuildScripts()
  File "/home/user/.local/lib/python3.10/site-packages/ambuild2/frontend/v2_2/context_manager.py", line 50, in parseBuildScripts
    self.execContext(cx)
  File "/home/user/.local/lib/python3.10/site-packages/ambuild2/frontend/v2_2/context_manager.py", line 152, in execContext
    exec(code, scriptGlobals)
  File "/very/long/path/on/hdd/data/Git/metamod-source/samples/s2_sample_mm/AMBuildScript", line 305, in <module>
    MMSPlugin.detectSDKs()
  File "/very/long/path/on/hdd/data/Git/metamod-source/samples/s2_sample_mm/AMBuildScript", line 123, in detectSDKs
    SdkHelpers.findSdks(builder, self.all_targets, sdk_list)
  File "/very/long/path/on/hdd/data/Git/metamod-source/samples/s2_sample_mm/../../hl2sdk-manifests/SdkHelpers.ambuild", line 23, in findSdks
    for sdk_name, sdk in SdkHelpers.getSdks(builder):
  File "/very/long/path/on/hdd/data/Git/metamod-source/samples/s2_sample_mm/../../hl2sdk-manifests/SdkHelpers.ambuild", line 126, in getSdks
    for sdk_manifest in os.listdir(sdk_manifest_dir):
FileNotFoundError: [Errno 2] No such file or directory: '/very/long/path/on/hdd/data/Git/metamod-source/samples/s2_sample_mm/hl2sdk-manifests/manifests'
Configure failed: [Errno 2] No such file or directory: '/very/long/path/on/hdd/data/Git/metamod-source/samples/s2_sample_mm/hl2sdk-manifests/manifests'

Header dependency parsing fails for GCC when using `-include`

If a project compiles with GCC and uses e.g. -include common.h on the compiler command line to auto-include a header on each source file that's compiled, then the AMBuild's hacky gcc -H-based header dependency parsing state machine in ParseGCCDeps fails to work properly.

GCC's -H option doesn't seem to print the header trace for -include'd header files. So not only does AMBuild miss dependency information, but part of the -H dump is inadvertently printed out to the user instead of being stripped as intended.

(ParseGCCDeps will be in state sFoundIncludeGuard, and then will run into one of the "multiple include guards may be useful for" filename lines that is not in its deps set, and so will enter the sIgnoring state prematurely.)

Maybe it's time to switch to a more proper gcc -Mx-based dependency parsing system? Perhaps using a pipe for the dependency "file"? It would certainly avoid many of the problems and ugliness of the current approach.

Add full rebuild option

AMBuild option that triggers full rebuild of the project.

Practical use case as of right now: there are numerous cases I've encountered myself, but not able to condense into a small repro case (thus file a proper bug report), where the build system doesn't hook up updated header files from an include directories, leading to having old debug and run time information about the structs/classes etc. At the very least I can tell that it mostly happen via working on metamod and updating the hl2sdk include files (specifically to name a few are convar.h, datamap.h), so if that could count as a repro case, then here you have it.

Also as a general note regarding having this feature available even if the idea behind ambuild is to cover all cases and have it detect relevant changes automatically etc, is that it's highly impossible to have a high percentage of these cases covered due to the limited/small usage of this build system by the public and not really highly active development of it, which leads to the cases above and more. So just as a general routine way of solving these cases and not waiting for a long time for the bug to be resolved, again due to the constraints of low public usage and low development interest, it's preferred in my eyes to have this option available.