I might be misunderstanding how this repo is meant to be used or missing a setup doc but it's not obvious to me how to use the regenerate tool to create a google form to put in front of someone or how I would go about applying it's output to a form I already have.

Not sure if this is an appropriate issue for this repo or should be spelled out in a different channel so feel free to push back.

I'd like to propose either the addition of a more advanced version of "Actively leaves space for others to share their ideas" or that we move it to be an advanced skill. I've observed (both as a manager and a engineer) that this is a very easy skill for the majority of new pivots because they don't have any opinions yet, or the confidence to share them.
However, it is a very hard skill for many pivots with a lot of experience. The impact of pivots with lots of experience and opinions leaving space for others is far greater than when new pivots do it, and I think it's crucial. I also don't think we capture that more advanced skill.
I'm not sure it's a P1 skill at all, because I haven't seen P1s who struggle with this. So my initial proposal is to move the skill to P3. However, I think it's valuable for all pivots to be thinking about, so I'm also open to adding a version of this skill at P3 which is worded slightly differently.

I find myself struggling to explain the difference between these two behaviors. In both cases the story is well-defined.

1. Is comfortable with carrying context on well-defined stories
2. Can onboard pairs into well-defined stories

When you carry-context, you naturally need to ramp your pair onto it.
Can someone clarify what the difference in behaviors this is meant to capture?

Many teams can be painful to be on, and painful to staff.

Some Pivots are able to help make their team more interesting and/or fun, and help recruit other Pivots to it.

For example, a Pivot may identify an interesting or exciting solution to address the pain, and build consensus for that solution. Others are excited about doing the work to reduce the pain, and are also very happy when it pays off.

Some Pivots may take it further and "market" the team to other Pivots as an interesting or exciting rotation opportunity. This helps Pivots self-select into solving certain problems, building alignment between the problems of the team and the allocated staff.

The impacts may be that:

• team morale goes up (e.g. as measured in Team Health Check)
• a team which was previously challenging to staff is now a desirable rotation
• other Pivots learn how to repeat this -- they learn to drive similar change elsewhere

I think this is a valuable skill set. How can we capture this it?

I would like to figure out a way to tie Engineering Personas into this areas of contribution repo. I still like using them in tandem to create a full picture of where my reports are going, and it'd be good to be able to talk about them in issues and be able to PR and contribute to them.

Currently, there is no automatic way for a change to this repo to propagate to the feedback forms that managers use to collect feedback for their reports.

In other words, if a PR is merged here, that change is only reflected here. In order for that change to affect how feedback is collected, individual managers would need to individually modify the feedback forms for each of their reports. This is a very high cost for change.

This issue is to track efforts related to reducing the cost of change to our feedback forms and heatmaps.

HTML5 has this sweet "details" thing, and Github Markdown supports it.

This could let us elaborate on the meaning of a skill, without making the definition super long

Changes to make:

• YAML skill definitions have a "details" block?
• Update markdown renderer

The document states that this is only for LA and SF people. Why is that?

What

We're considering capturing the skill definitions in a set of YAML files in this repo. To see some examples of what we're thinking, check out this branch.

This issue is to document our reasoning, and invite comment!

Why

To make progress on #11, we need to build tools that can work with skill definitions. If the skills were defined in a structured format, it would be easier to develop those tools. For example, to:

a. auto-generate nicely-formatted Markdown or other browsable documentation
b. auto-update Feedback Forms

FAQ

1. Why YAML and not <alternative>?
   We're considering using YAML format because it is relatively easy for humans to read and write (in small quantities), and is already familiar to most of the users of this repository.

2. Why do we need a structured format?
   It isn't strictly necessary for building automation for the Feedback forms, but that automation would need to be somewhat more complex and brittle if it had to first parse the existing Markdown files. If we wanted to change the visual display of those Markdown files (e.g. to address #9), we'd need to do extra work to not break that parsing behavior.

3. Are you sure?
   We've done some "user interviews" to try out a YAML-centric workflows for contributing to this repository. They went okay. Also, this kind of thing is relatively easy to change later if we really don't like it.

4. Other questions or feedback?
   Please leave comments in this issue!

Hi - currently there's a areas.yaml and skills.yaml, but I was wondering if you'd consider having a single file per area which then includes the skills?

• It seems like skills are always in relation to a single area anyways.
• It would help keep the context between an area and a skill closer together.
• Areas can sometimes be within a specific context/team, so it would allow copy/paste/tweaking on a file level to capture the "area" + skills.

If interested, I'd be happy to PR the change.

As a manager, given I have a spreadsheet or form where I messed-up a skill definition (e.g. fat-fingered some text somewhere), I'd like to be informed of this error and have an option to fix it.

Note: We wanted to do this as part of the soon-to-wrap-up manager initiative, but didn't complete it.

Pivotal provides the Gitbot service to synchronize issues and pull requests made against public GitHub repos with Pivotal Tracker projects.

If you do not want to use Pivotal Tracker to manage this GitHub repo, you do not need to take any action.

If you are a Pivotal employee, you can configure Gitbot to sync your GitHub repo to your Pivotal Tracker project with a pull request.

Steps:

• Add the Toolsmiths-Bots team to have admin access to your repo
• Add the cf-gitbot ([email protected]) user to have owner access to your Pivotal Tracker project
• Create new branch in this repo: cfgitbot-config (an ask+cf@ ticket is the fastest way to get write access if you get a 404)
• Add your new repo and or project to config-production.yml file
• Submit a PR, which will get auto-merged if you've done it right. Detailed steps here

If you are not a pivotal employee, you can request that [email protected] set up the integration for you.

You might also be interested in configuring GitHub's Service Hook for Tracker on your repo so you can link your commits to Tracker stories. You can do this yourself by following the directions at:

https://www.pivotaltracker.com/blog/guide-githubs-service-hook-tracker/

If there are any questions, please reach out to [email protected].

The change in #43 doesn't fully apply to the form.

What's missing:

• Area title on the first-page dropdown menu
• Area description on checkbox pages

What seems to work:

• Area title on checkbox pages
• Area title for additional context box
• Spreadsheet

@ctaymor recently gave me feedback about one of my reports and how the IC effectively kept the team productive during a support escalation. Specifically, my report exhibited knowledge of linux tools and effective debugging skills that prevented the team from chasing red herrings:

• Used his technical knowledge to help steer the team towards a better outcome as they debugged the customer environment.
• Suggested tools for debugging that the team had forgotten about (tool knowledge)
• Pointed out when certain approaches wouldn't solve the problem at hand (troubleshooting/debugging skills)
      * e.g. Let's not try to debug the network, because logs are showing that the networking connectivity wasn't preventing nodes from connecting (ruling out possible causes)
      * e.g. suggested rotating logs to avoid red herrings (making sure data is relevant and accurate)

We met to try to encode the feedback in the form and had a hard time finding anything that could capture this feedback. The support skill area seemed closest, but focused on prioritization of support tickets, looping in stakeholders, communicating effectively with customers, and improving the team's process with regards to support. Because the Pivot's technical knowledge was a major factor, we looked in technical execution and couldn't find too much there either, since the focus of that skill area is mostly around execution on stories and driving improvements to codebases.

I've got a few questions:

• Have other managers had a hard time collecting and encoding feedback for these kinds of skills?
• Do people feel like this is worth capturing in the heatmap? If so, which skill area should it be a part of?

Github's default width for rendering markdown is not very wide and some of the tables with a lot of content end up getting cut off on the right (e.g. https://github.com/pivotal-cf/areas-of-contribution/blob/296124f22aaba971aa1f51fef82ff386a166a232/cf-domain-expertise.md).

Could we flip the orientation of the tables or make some other formatting changes so as not to hide higher p-level content?

The domain of our products is larger than "Cloud Foundry." Within the Pivotal Cloud R&D organization we're now shipping products that aren't directly tied to "Cloud Foundry" and we're finding that our engineers impact teams and customers through their expertise in domains such as Kubernetes and Istio.

Is there value in keeping the letters "CF" in this skill area?

I've seen multiple situations where people thought they submitted the form but didn't because they chose "Submit Form", clicked "Next" and then closed the window without clicking the native "Submit" button.

If we remove the ability to include multiple areas in a single submission the dropdown in question won't be needed anymore.

This is an important skill. Maybe an arc?

• p1: is comfortable letting go of a priority that they have worked on
• p2: recognizes when their pair or team would benefit from cutting scope and advocates for it
• p3: effectively advocates to anchor/PM to cut scope or stop efforts that no longer align with team's priorities
• p4: collaborates with leadership to identify cross-team scope that should be cut, builds alignment with stakeholders, and drives it to completion
• p5: regularly collaborates with leadership to align engineering efforts with strategic needs of the organization

In October 2018 Mike Dalessio sent an email to the CF Engineering Managers announcing a P-Level Skills Chart for CF Engineering.

Since this was announced after this repo was created the relationship between the contents of this repo & the P-Level Skills Chart for CF Engineering isn't clearly articulated anywhere. As a consequence they can seem like alternate sources of truth and disconnected.

Another interpretation is that there's an opportunity for these to be 2 views of the same information, varying simply by their degree of detail & level of abstraction. This is the view I'd like.

An approach to get to this might be:

• An exercise to compare the contents of this repo with the contents of the skills chart
• Write a description of their intended relationship
• Subsequent exercises to remove any dissonance based on their intended relationship

P2: Understands and explains why balanced teams and balanced decision-making is important

Though I think that this is something I hope a P2 would do in the future, I don't believe our current organization and team structures are set up to enable this as a P2 activity. This seems like something that would currently be done by P3+ who's in the process of advocating for a balanced team.

Until both product and engineering are aligned on all or most CF teams about the benefits of a balanced team, I would not say most teams don't have the space nor opportunities to discuss this in a way that would allow a P2 to exhibit this behavior.

I feel that we should revisit this question. There is mounting evidence that rotating for rotation's sake may not be helpful.

Perhaps this is better worded as: "Is comfortable pairing with everyone on the team" or replaced entirely. I'm not certain what the original driver for this question is.

Problem

When collecting feedback from an engineer's PM or LL, the set of relevant questions are often spread across several skill areas, and may not be immediately discoverable.

Alternately, there are some skill definitions that are rarely observable by anyone other than a Pivot's manager.

Possible solution

One benefit of the new yaml format (#23) is that individual skills may be annotated with additional metadata.

For example, right now we've got

  - id: sb975bf39
    description: >-
      Is aligned with the priorities of the organization, and approaches their team allocation as a
      collaborative effort with leadership.
    area: alignment
    level: p4

but we could easily add new, optional key: value pairs that encode other useful information.

For example:

  - id: sb975bf39
    description: >-
      Is aligned with the priorities of the organization, and approaches their team allocation as a
      collaborative effort with leadership.
    area: alignment
    level: p4
+   observable_by: [manager, LL]

Another example:

  - id: s5c7c3275
    description: "Pairs with PM+Design on technical solutioning (eg: in pre-IPM, preparing for inception, etc.)"
    area: project-leadership
    level: p2
+   observable_by: [PM, anchor]

We could easily provide "filtered" views of the skills that would help with "what questions should I ask their LL?"

Thanks to @dsabeti for the idea!

There are a set of skills that are helpful in making change within an organization.

They involve identifying problems, prioritizing, getting buy-in, framing experiments, identifying risks and mitigations, communicating about the effort with varying circles of stakeholders, etc.

Maybe this is it's own skill area, maybe it is some kind of "theme" that cuts across "Engineering Process", "Collaboration" or "Project Leadership."

cc @ragaskar

I have the experience that an team members can only observe how fast their team member has ramped up on their current team. Whether or not a person "can ramp up" on most CF teams is hard to asses if they haven't had a chance to observe it. I also worry that it can lead the conversation into subjective assessment of potential rather than demonstrated behavior. Managers and reports already know what teams they have rotated through and how quickly they ramped up.

$ grep -ri opinion areas-of-contribution/

nada.

I'm surprised we don't have anything about developing opinions, sharing opinions, engaging with others about their opinions, holding them loosely, etc.

Thanks to @rodolfo2488 for pointing this point.

P2: Surfaces and discusses various factors (outlined in P3+) when approaching a technical decision, with support from other pivots to make the final decision.

This has added cognitive overhead to my feedback collection, as the next page is not visible. We should either make the factors explicit on this page or reword this to be more accurate.

Working with @rainmaker now, we discovered that the template links in our users guide point at versions of the data before the migration to AoC v0.2.0.

We should migrate the templates up so that managers with new reports don't need to manually do a migration at the start.

This one seems very vague and almost too passive. It's hard to imagine that anyone on the team isn't in some way learning how other teams/customers are using their product just by virtue of listening in meetings etc. Maybe if it was "Has sound knowledge about how" or something it would feel a little easier to say when someone is missing this skill.

https://github.com/pivotal-cf/areas-of-contribution/blame/bbd2f84477458de6f208ada3925b0961537f27bc/cf-domain-expertise.md#L26

"Low Impact" & "High Impact" is confusing for first-time users for the form.

It can sound like "Good" (+1) and "Best" (+2) instead of "got worse" (-1) and "got better" (+1). It might be clearer for first-timer users if we rephrased this to something like "Impact: Better" And "Impact: Worse", which is consistent with the description in the readme.

I was going over the feedback form with someone, and they asked what "Can onboard pairs into well-defined stories" (P2) meant, and how it differed from "Is comfortable with carrying context on well-defined stories" (a P1 skill). I wasn't sure myself.

I tend to think of it as "can not just do a minimal job carrying context, such that your team is ok with you carrying context when rotating pairs, but also give more of this history of how the story evolved, and where you're going". Sort of a "keeping context well".

I'd like to clarify the language on this skill, but first I want to understand how others interpret this skill, to make sure I'm in alignment with other managers.

I was wondering if it is alright for managers to share the name of the person who gave a certain piece of feedback. @ryantang suggested we can add a question to the form before "submit" asking if they are comfortable with us sharing their name with the report. Do y'all think it will be a valuable prompt to be added to the form?

Motivation

In order to reduce the cost of change to this repo (#11), a skill definition needs to be changeable. For some kinds of change, like a typo fix, the feedback form responses collected against the original definition should be carried forward to the new definitions. In other cases, like deleting a skill definition entirely, or substantially changing the meaning of a skill definition, the responses might be preserved somehow, but shouldn't be recorded as applying to a new skill definition.

The author of a change to a skill definition ought to be able to define how the response data to the original definition should be handled. This would be specifying a "migration."

Below we propose how a contributor to this repo might specify a migration, using a simple YAML syntax.

To update or not to update

There are 3 types of change:

• create
• delete
• update

Feedback responses to deleted skills are archived, but not migrated anywhere.

Conversely, an update results in the feedback responses being migrated to the new skill definition.

Schema examples

We'll illustrate the proposed schema with a few examples

1. Typo fix. Preserve existing responses.

   changes:
   - type: update
     # typo fix delviery --> delivery
     old:
       area: Technical Execution
       level: P4
       definition: "Can navigate their way through legacy systems and improve throughput of the team (eg: notices complex code paths are slowing down feature delviery, facilitates conversations with the team on how to simplify them, gets buy-in from PM+leadership to prioritize this work, drives it to completion with the team.)"
     new:
       area: Technical Execution
       level: P4
       definition: "Can navigate their way through legacy systems and improve throughput of the team (eg: notices complex code paths are slowing down feature delivery, facilitates conversations with the team on how to simplify them, gets buy-in from PM+leadership to prioritize this work, drives it to completion with the team.)"

2. Change the P-level associated with a skill definition. Preserve existing responses.

   changes:
   - type: update
     old:
       area: Technical Execution
       level: P2
       definition: "Discusses the balance of short term execution with long term health"
     new:
       area: Technical Execution
       level: P3
       definition: "Discusses the balance of short term execution with long term health"

3. Change the meaning of a skill definition and its P-level. Do not apply old responses to the new skill.

   changes:
   - type: delete
     old:
       area: Technical Execution
       level: P2
       definition: "Discusses the balance of short term execution with long term health"
   - type: create
     new:
       area: Technical Execution
       level: P3
       definition: "Calibrates pace of execution to balance short-term delivery with long-term health"

4. Split up a single skill definition into multiple definitions. Do not apply old responses to new skills.

   changes:
   - type: delete
     old:
       area: Technical Decision-Making
       level: P2
       definition: "Surfaces and discusses various factors (outlined in P3+) when approaching a technical decision, with support from other pivots to make the final decision."
   - type: create
     new:
       area: Technical Decision-Making
       level: P2
       definition: "Surfaces and discusses maintainability when approaching a technical decision, with support from other pivots to make the final decision."
   - type: create
     new:
       area: Technical Decision-Making
       level: P2
       definition: "Surfaces and discusses support when approaching a technical decision, with support from other pivots to make the final decision."
   - type: create
     new:
       area: Technical Decision-Making
       level: P2
       definition: "Surfaces and discusses performance when approaching a technical decision, with support from other pivots to make the final decision."
   ...

Data migration

• type: update will modify an item. It will always preserve existing response data
• changes that affect the meaning of an item should not be expressed as an update but instead as a delete followed by an insert. This will cause existing response data to not be migrated to the new item.

Feedback?

Please leave thoughts and questions below!

This is a follow-up to #21. Same problem statement, different proposed user-interface.

Problem statement

(same as #21)
In order to reduce the cost of change to this repo (#11), a skill definition needs to be changeable. For some kinds of change, like a typo fix, the feedback form responses collected against the original definition should be "migrated" to the definition. In other cases, like deleting a skill definition entirely, or substantially changing the meaning of a skill definition, the responses might be archived somehow, but shouldn't be recorded as applying to new skill definition(s).

When a contributor opens a PR here proposing a change, they ought to be able to specify whether response data should be migrated or not.

Proposed solution: Git is the source of truth

A contributor opens a PR with a commit that changes some text in the skills.yaml file.

The decision to migrate or not is conveyed by whether or not the id field of a skill definition remains intact or is lost.

Contributor decision tree

• Should prior responses be migrated to the new skill definition?
  • yes?
    • then keep the id field intact
    • it is ok to mutate description and any other fields
  • no?
    • then change the id field (in other words, "delete and re-recreate")
    • ok to change any fields

Examples

1. Typo fix. Preserve existing responses.

    - id: sf53e8688
      description: >-
        Can navigate their way through legacy systems and improve throughput of the team (eg: notices
   -    complex code paths are slowing down feature delviery, facilitates conversations with the team on
   +    complex code paths are slowing down feature delivery, facilitates conversations with the team on
        how to simplify them, gets buy-in from PM+leadership to prioritize this work, drives it to
        completion with the team.)
      area: technical-execution

2. Change the P-level associated with a skill definition. Preserve existing responses.

    - id: s2be43833
      description: Discusses the balance of short term execution with long term health
   area: technical-execution
   -  level: p2
   +  level: p3

3. Change the meaning of a skill definition and its P-level. Do not migrate old responses to the new skill.

   -- id: s2be43833
   -  description: Discusses the balance of short term execution with long term health
   +- id: s8131ec9e
   +  description: Calibrates pace of execution to balance short term delivery with long-term health
      area: technical-execution
   -  level: p2
   +  level: p3

Sometimes a set of skills forms a progression from P1 to P4.

Can we represent this somehow?

Possible changes:

• New file, arcs.yaml that lists the arcs, gives each an id and description
• Tag some skills with one or more arcs
• Markdown rendering shows the arcs associated with each skill

Our automated tooling can currently update the text for skills and areas, but it doesn't support changing the number, order or titles of the columns in the feedback form.

To support automated updates to forms and sheets, we should enhance the tooling with these capabilities.

The description of the repo is not very inclusive of non US people.

If you live in Niigata, are you on the west coast?

Sometimes when I'm collecting feedback the reviewer will note that "we don't really do that on this team" or "they haven't really had an opportunity to do that." The inception-related questions often get this response, as well as the story and acceptance criteria writing questions.

I might like to be able to mark "not observable" on a skill and then visualize on the heatmap if a certain percentage skills had that response in an area, but I don't know if the system is already designed to take this into account somehow.

How should we take it into account with leveling if someone appears weak in an area because there team doesn't have many opportunities to exercise the relevant skills?

Currently the text on the "Advanced" form page is identical to on the first page.

There may be value in having different text there, to better contextualize some of those advanced skills.

Tooling and YAML format would need to be updated to support this kind of change.

I find this question somewhat ambiguous.
If it is "Is disciplined in personally following the team's engineering process", which I view as a P1 skill.
Or if it is meant to imply
"Notices when there is a deviation from the team's engineering process and holds the team accountable" which would be P2 or P3.

We should add some guiding principles to provide additional context on how the areas-of-contribution is designed and to enable the community to easily make changes while preserving the bigger picture design and intent.

This is one of the early technical execution questions. New Pivots especially often don't understand what this means.

I tend to describe it as "You would know it if they weren't doing it. Are they present in the pair? Are they engaged with what's happening and asking questions?"

Does that fit with other folks understanding of this question? Have you also seen feedback-givers get tripped up on this one? Is there a way we could rephrase to make this clearer?