In Bazel, a resource or structured_resource can very well be an artifact. In the case of XCHammer->XCHammer. For a good test case of this, the TulsiGenerator framework builds a macos_command_line_application and bundles it inside of the app.

XCHammer was originally designed to produce a great user experience for local on Xcode development. However, building with Bazel is still useful and widely preferred.

The current approach to achieving this is not very maintainable, documented, or ideal:

• Create an Xcode workspace
• Put XCHammer projects in that workspace
• Create a separate Xcode project and targets that run bazel ( i.e. a Tulsi gen'd project with no files )

Come up with a way to integrate this into XCHammer.

Hammer.bzl currently depends on these load statements

load(
    "@build_bazel_rules_apple//apple/bundling:entitlements.bzl",
    entitlements_rule = "entitlements",
)
load(
    "@build_bazel_rules_apple//apple/bundling:entitlements.bzl",
    "AppleEntitlementsInfo",
)

These have been moved into internal rules and probably shouldn't be depended on anymore. We should revisit if our custom entitlement logic is still necessary given recent work on rules_apple

Setup a CI system that runs the existing integration tests.

Is XCHammer intended to pass a -c/--compilation_mode flag based on Xcode's build configuration?

In Xcode, the app target has "Debug" as its build configuration, but this appears to not be reflected in the bazel build invocation. I had to add -c dbg to the config in order to get a debug build.

I'm not sure how relevant this is, but Tulsi's has a doc that says it sets --compilation_mode based on Xcode.

However, I don't see either -c or --compilation_mode string in build_bazel.py.

Codesigning is failing on iOS builds for Xcode builds:

App installation failed: A valid provisioning profile for this executable was not found.

On a first look, it looks like we are missing the embedded.mobileprovision in the built .app.

Here is an example diff of what happens today.

       <BuildableReference
             BuildableIdentifier = "primary"
-            BlueprintIdentifier = "NT_217978137037"
-            BuildableName = "share-extension-Bazel.appex"
+            BlueprintIdentifier = ""
+            BuildableName = "share-extension.appex-Bazel"
             BlueprintName = "share-extension-Bazel"
             ReferencedContainer = "container:UrlGet.xcodeproj">
          </BuildableReference>

If you open the project in Xcode it'll fix this for you but we should be generating the buildable name for Bazel schemes.

In my xchammer.yml, I listed an abbreviated label under targets, for example //Modules/User instead of //Modules/User:User. This resulted in no User-Bazel target, because it was looking for that label in the list of all targets, but the list of all targets uses a full label.

XCHammer should either support abbreviated labels, or generate an error when one is used.

For our project, I am wondering whether to use module paths instead of Bazel labels. For people new to bazel, Modules/User is natural but //Modules/User:User is foreign.

We currently check in the result of running the generator on the sample project ( sample/UrlGet ). This has an unfortunate effect of checking in absolute paths of the person who last ran make run into the repo.

Consider coming up with a way to stabilize this so we can vet changes to XCHammer's output without the noise.

I've got a an ios_unit_test rule containing a test dependency in an external Xcode project. It looks like this test dependency went missing in a recent version of XCHammer.

An outcome of fixing this should be adding an example of this ( and workspace mode ) that is tested on CI.

We use a hash to determine if a project should be generated or not.

There are some circumstances where having the deps hash in an external file may not be ideal.

For example, when the user is prompted by Xcode the project has changed. If the user selects Keep Xcode Version, then they will end up with an incorrect project.

Writing the hash into the project ensures the hash always matches the Xcode project in obscure cases.

Currently we add an empty open/close tag command line options when we have zero options to specify. We should avoid creating this entirely.

      <MacroExpansion>
          <BuildableReference
             BuildableIdentifier = "primary"
-            BlueprintIdentifier = "NT_217978137037"
-            BuildableName = "share-extension-Bazel.appex"
+            BlueprintIdentifier = ""
+            BuildableName = "share-extension.appex-Bazel"
             BlueprintName = "share-extension-Bazel"
             ReferencedContainer = "container:UrlGet.xcodeproj">
          </BuildableReference>
       </MacroExpansion>
+      <CommandLineArguments>
+      </CommandLineArguments>
       <AdditionalOptions>
       </AdditionalOptions>

The generated project contains only the -Bazel targets in the active scheme. The Xcode targets exist, but are not in the scheme, so can't be built without manually adding them to the scheme. Note these are swift_library targets (static libraries).

Currently XCHammer creates Bazel targets by the labels specified in the XCHammer config.

Since we do a naive string comparison, it will miss labels that aren't using the full label syntax:

            return allXCTargets
                .filter { specifiedLabels.contains($0.value.xcodeTarget.label) }
                .map { $0.value.xcodeTarget }

This command results in files like /usr/local/bin/Info.plist and /usr/local/bin/StubIOSAppExtensionInfoPlist.plist.

ditto tmp_build_dir/XCHammer /usr/local/bin/

There are certain parts like platform architectures that are not stable and causes noise in PR reviews and unnecessary diffs. We should ensure we sort any collection types before rendering this file

Here is an example:
https://github.com/pinterest/xchammer/pull/84/files#diff-31666cd9f1523b974543b5f819d5665eR137

XCHammer creates a hash of inputs to a project to determine if a project should be generated.

Consequently, the hash must be computed before the project is generated. Today, this hash value is computed both before and after the project is generated. Consider reusing the hash to avoid computing it twice.

        // in Sources/XCHammer/Generator.swift
        // TODO: reuse the hash from the invalidation phase
        updateDepsHash(genOptions: genOptions, projectPath: tempProjectPath)

Ran into this error after bumping XcodeGen.

At first look, xcodegen is duplicating symlinked files in a copyfiles phase.

Build system information
error: Multiple commands produce '/Users/jerry/Library/Developer/Xcode/DerivedData/UrlGet-evlfsbrrogjoftawvtpkwphhiltc/Build/Products/Debug-iphonesimulator/include/PINCache-Arc-exception-safe-ios-12.0/PINCache.h':
1) Target 'PINCache-Arc-exception-safe-ios-12.0' (project 'UrlGet') has copy command from '/Users/jerry/Projects/xchammer-github/IntegrationTests/Sandbox/UrlGet/Vendor/PINCache/Source/PINCache.h' to '/Users/jerry/Library/Developer/Xcode/DerivedData/UrlGet-evlfsbrrogjoftawvtpkwphhiltc/Build/Products/Debug-iphonesimulator/include/PINCache-Arc-exception-safe-ios-12.0/PINCache.h'
2) Target 'PINCache-Arc-exception-safe-ios-12.0' (project 'UrlGet') has copy command from '/Users/jerry/Projects/xchammer-github/IntegrationTests/Sandbox/UrlGet/Vendor/PINCache/pod_support/Headers/Public/PINCache/PINCache.h' to '/Users/jerry/Library/Developer/Xcode/DerivedData/UrlGet-evlfsbrrogjoftawvtpkwphhiltc/Build/Products/Debug-iphonesimulator/include/PINCache-Arc-exception-safe-ios-12.0/PINCache.h'

@keith mentioned today that bazel-genfiles will soon unified with bazel-bin. XCHammer will have to be updated accordingly. See

1. bazelbuild/bazel#6761
2. --incompatible_merge_genfiles_directory in https://docs.bazel.build/versions/master/command-line-reference.html

We should potentially include all entitlements for GeneratedFiles when generating multiple projects.

For integration tests, it seems to be missing some - specifically the test host's entitlements.

To match Bazel's behavior, consider building extensions as part of a given application. Ideally this is part of target dependencies.

It'd be awesome to integrate XCHammer project generation with Bazel. This would allow remote caching of Xcode projects using the existing Bazel infrastructure, which could speed up workflows like rebasing master and branch switching.

Considerations:

• Running Aspects: With the ability to run aspects from a rule, this should be technically possible. It'd be nice if we could write a rule that uses the output of aspects as a input. It might be worth looking into cquery too.
• Run Bazel queries: we'd want to run the query as part of the rule as well.
• Xcode projects are portable across machines: paths, noop inputs, are the minimum.
• Multi project outputs: with N > 1 projects, it'd be ideal to output 1 Xcode project per Bazel target.

I am trying to access the main app's xcasset catalog from my Snapshot Test target, by setting the main app as the dependency target of my test target, but it didn't work, i.e. the xcasset catalog would not show up in my test target's resources list.

It looks like we have the CLI code to read this argument but it appears to not be used throughout the codebase. Potentially this functionality was lost during recent cleanup?

I ran make workspace and got a build failure in Tulsi, see the full log below. From Tulsi's checkout directory I tried xcodebuild clean and that too fails (same error message, but empty stack trace).

I realize this might not be XCHammer's error to own, but it does block workspace generation.

Error message:

API VIOLATION: rock_XPCObjectToNSDictionary would have returned nil without an error set.

Erro stack:

2018-09-17 14:00:19.774 xcodebuild[83737:3283066] *** Assertion failure in NSDictionary * _Nullable rock_XPCObjectToNSDictionary(xpc_object_t  _Nonnull __strong, ROCKSessionManager * _Nullable __strong, NSError * _Nullable __autoreleasing * _Nullable)(), /BuildRoot/Library/Caches/com.apple.xbs/Sources/ROCKit/ROCKit-24/ROCKit/NSDictionary+ROCK.m:172
** INTERNAL ERROR: Uncaught exception **
Uncaught Exception: API VIOLATION: rock_XPCObjectToNSDictionary would have returned nil without an error set.
Stack:
  0   __exceptionPreprocess (in CoreFoundation)
  1   objc_exception_throw (in libobjc.A.dylib)
  2   +[NSException raise:format:arguments:] (in CoreFoundation)
  3   -[NSAssertionHandler handleFailureInFunction:file:lineNumber:description:] (in Foundation)
  4   rock_XPCObjectToNSDictionary (in ROCKit)
  5   __50-[SimDeviceSet subscribeToNotificationsWithError:]_block_invoke (in CoreSimulator)
  6   xpc_array_apply (in libxpc.dylib)
  7   -[SimDeviceSet subscribeToNotificationsWithError:] (in CoreSimulator)
  8   __45-[SimServiceContext deviceSetWithPath:error:]_block_invoke (in CoreSimulator)
  9   _dispatch_client_callout (in libdispatch.dylib)
 10   _dispatch_queue_barrier_sync_invoke_and_complete (in libdispatch.dylib)
 11   -[SimServiceContext deviceSetWithPath:error:] (in CoreSimulator)
 12   -[SimServiceContext defaultDeviceSetWithError:] (in CoreSimulator)
 13   __42-[DVTiPhoneSimulatorLocator startLocating]_block_invoke_2 (in IDEiOSSupportCore)
 14   __DVTDispatchAsync_block_invoke (in DVTFoundation)
 15   _dispatch_call_block_and_release (in libdispatch.dylib)
 16   _dispatch_client_callout (in libdispatch.dylib)
 17   _dispatch_continuation_pop (in libdispatch.dylib)
 18   _dispatch_async_redirect_invoke (in libdispatch.dylib)
 19   _dispatch_root_queue_drain (in libdispatch.dylib)
 20   _dispatch_worker_thread3 (in libdispatch.dylib)
 21   _pthread_wqthread (in libsystem_pthread.dylib)
 22   start_wqthread (in libsystem_pthread.dylib)

zsh: abort      xcodebuild build -verbose -scheme TulsiApp -derivedDataPath  -project   

An example message is:

(reformatted for readability)

:: error: Failed to find an output artifact for target ~/Library/Developer/Xcode/DerivedData/HammerLyft-apdrvgdqsckgxbeluodmnicxohsd/Build/Products/Debug-iphonesimulator/libArchitectureCore-Bazel.a
in output map {
    u'generated_sources': [
        [u'bazel-out/ios_x86_64-fastbuild/bin/Modules/ArchitectureCore/ArchitectureCore-Swift.h', u'Modules/ArchitectureCore/ArchitectureCore-Swift.h'],
        [u'bazel-out/ios_x86_64-fastbuild/bin/Modules/ArchitectureCore/ArchitectureCore.swiftmodule', u'Modules/ArchitectureCore/ArchitectureCore.swiftmodule'],
        [u'bazel-out/ios_x86_64-fastbuild/bin/Modules/ArchitectureCore/ArchitectureCore.modulemaps/module.modulemap', u'Modules/ArchitectureCore/ArchitectureCore.modulemaps/module.modulemap']
    ],
    u'has_dsym': False
}

Currently the XCScheme struct takes in the commandLineArguments for cli args but not the environment variables (key-value pairs). Would be great to have this support so we can pass in the env var pairs for each target in the XCHammer.yaml file under targetConfig

It is certain that XCHammer works on a subset Bazel versions and the integration test suite should prove that.

If a user provides a version number outside of our known working versions, consider printing a non fatal warning.

I have a unit test target in an XCHammer generated project. In the scheme that's created, I would expect the test target to be added to the scheme's test action's Tests section, but it is not.

In order to enable the Product > Test action in Xcode, I must manually go into the scheme editor, choose the "Test" action in the left hand pane, choose the "Info" tab, and add the UnitTest target to the Tests section manually.

This currently happens at least in bazel_build_settings.py but we should audit the entire project for absolute file paths. This will be a blocker as we start to think about project caching and adds noise to diffs

Example here:
https://github.com/pinterest/xchammer/pull/84/files#diff-31666cd9f1523b974543b5f819d5665eR133

Hello!

I am trying to play with xchammer. I followed the only documentation that I found in the repo, cloned it and ran make install. The compilation of the project and its dependencies works as expected, but ends up failing with ditto: can't get real path for source 'tulsi-aspects'. You can see the last part of the log here:

...
Compile Swift Module 'Commandant' (9 sources)
Compile Swift Module 'AEXML' (5 sources)
Compile CYaml src/emitter.c
Compile CYaml src/api.c
Compile Swift Module 'Yams' (16 sources)
Compile Swift Module 'xcproj' (61 sources)
Archiving /Users/balestrapatrick/Downloads/xchammer-master/.build/x86_64-apple-macosx10.10/release/libTulsiGenerator.a
Compile Swift Module 'ProjectSpec' (25 sources)
Compile Swift Module 'XcodeGenKit' (8 sources)
Compile Swift Module 'XCHammer' (20 sources)
Linking ./.build/x86_64-apple-macosx10.10/release/XCHammer
# Install bundle resources
# Install Tulsi resources
# Tulsi utilizes NSBundle heavily. All assets need to exist at the root of
# this directory in order for NSBundle and Tulsi's searching logic to work
# in the context of our custom release package.
ditto: can't get real path for source 'tulsi-aspects'
make: *** [build-impl] Error 1

Any idea what's wrong? I have Tulsi installed, and from what I can understand the Makefile tries to reference tulsi-aspects which is unsuccessful on my machine (not sure exactly what tulsi-aspects is and couldn't find many information about it).

Thank you! 👍

When I run XCHammer, I'm no longer seeing errors from the aspect invocation

After running make workspace the generated Xcode project for XCHammer it's self, doesn't have this flag.

We should eventually use XCHammer to build and develop XCHammer, which will fix this issue.

Repository rules are a feature of bazel that have quite different conventions than code within the workspace root directory.

For the most part, it should be pretty straight forward to add this support. We currently assume all paths are relative to the workspace root.

Symlinking external into the workspace root may be an easy way to achieve this since all of the file paths reference external/ in Bazel.

Allow the user to set targets on a run profile test or build action of a scheme through XCHammerConfig.

Improve CLI messages for workspace generation.

Consider adding:

• colors to indicate failures or success
• emojis to give it a fun and friendly feel
• print the name of the workspace we are generating
• print the path to the config file and settings we are using

and, other things that make it easier to understand what XCHammer is doing.

The Apple rules have deprecated these framework / resource rules so we need to make sure we need to support the new versions as well (if we don’t want this to be a breaking change)

Notice: This is the last version that supports objc_framework, objc_bundle and objc_bundle_library. The next version of rules_apple will not support these rules. Please migrate to their Starlark counterparts apple_dynamic_framework_import/apple_static_framework_import, apple_bundle_import and apple_resource_bundle.

Since the aspects, rules and bazel capabilities are constantly changing it would be good to be really clear on what version people should be running

We currently generate projects during the build if needed. In addition, it'd be awesome to have a program that watches for changes in files, and updates projects for me.

Any reason to not allow XCHammer generate without a config file name, having it use XCHammer.yml if present?

I tried to add a defaultValue to the Argument in main.swift, but that resulted in a Commandant can of worms.

We need this for the Xcode targets to work

Currently we create the bundle targets but they need to be dependencies of top-level targets and copied into the final bundle. Currently this does not happen.

When XCHammer was originally implemented, the goal was to version program along with each commit. The point of versioning every tooling dependency is to ensure that a given commit can build: I don't want to checkout a commit and not have the right version of X program..

Currently, this is achieved by checking in the binary. At a few MB, it really isn't that big of a deal, but we could do better.

Consider how this approach will work:

1. Do trusted CI builds for each commit
2. Github binary releases for each build
3. Load the binary with other systems like nix and Bazel

All of these may have implications on runtime performance of builds, as we run XCHammer constantly. Consider how this change would manifest end to end. Take an opinionated stance, don't force a process on users, but make it easy and documented.

XCHammer is tested on custom modules. The modulemaps generated in Bazel are somewhat of special case: consider how to handle these.

On the latest master build of XCHammer, a large cocoapods Xcode project is taking: 477 seconds to generate. Prior it was around 10 seconds.

This is a blocker to running on HEAD of XCHammer. Come up with a way to exemplify this condition in the project and on the CI going forward.

Not sure how this should be handled, but Readmes are not included in the final xcode project for the libraries. They are a useful documentation source and should be visible within xcode.

The following directory structure does not get built into the xcodeproject as a localised resource:

There was a hard filter to prevent plists from being ingested as resources.

There are three genrules in our build. These do a bit of codegen, and localization setup. The genrules look something like this:

genrule(
    name=name + "NameSuffix",
    srcs=[...],
    cmd="$(location //tools:some_script) generate $< $@ " + name,
    outs=["FilePrefix+{}.swift".format(name)],
    tools=["//tools:some_script"],
)

These genrules are called from macros of our own. The name refers to a module.

Running XCHammer generate xchammer.yaml gives the following errors:

swiftException(XCHammer.Generator.GenerateError.some(184 Spec validations errors:
    - Target "AppCoreUI" has a missing source directory "/Users/me/src/TheApp/Modules/AppCoreUI/AppCoreUIViewControllers.swift"
    - Target "AppCoreUI" has a missing source directory "/Users/me/src/TheApp/Modules/AppCoreUI/I18N+AppCoreUI.swift"
    - Target "AppCoreUIResources" has a missing source directory "/Users/me/src/TheApp/Modules/AppCoreUI/Localizable.strings"

Here you can see the outs of the genrules are causing problems. Note also the strange missing source directory, not sure why these files are being called directories.

What more info can I provide?

XCHammer needs a value for SWIFT_VERSION. Currently it runs xcrun swift package tools-version (see #85) but this only works for projects that have a Package.swift.

https://github.com/pinterest/xchammer/blob/c96f82173572b73544df435e4b0015784e5aecdc/Sources/XCHammer/XcodeTarget.swift#L604

I'm not sure what the best way is to get a swift version. Maybe the output of swift -version, but that doesn't looks to be ad hoc text, and could potentially change format and break XCHammer.