Chapter 3 Standard Operations

3.1 Team PSD

Meet the members and partners of Team Participatory Systems Dynamics at mtl.how/team.

3.2 Scientific Values

Team PSD Scientific Values guide additional Participatory and Open Science principles:

  • Participatory Research encourages us to co-create our scientific research. Therefore…
  • We share decisions, which requires a high level of documentation.
  • We seek greater equity among partners in how collaborate, which requires responsive pivots with new stakeholder inputs.
  • We strive use transparent and accessible processes and platforms, and develop transparent, accessible resources.

3.3 Guiding Principles

  • Value Streams: All
  • Workstreams: All
  • Workflows: All

Teams Channel Link

3.3.1 Open-Source, Transparent, Reproducible

We value an open-source, transparent & reproducible workflow.

  • Most of our work is public-facing with the exception of any items that include Protected Health Information (PHI) or Personally Identifiable Information (PII). All of our public materials are free to download and use. We want our insights and resources to benefit the larger community.
  • We use R, a free open-source coding language and a specific file naming convention to ensure all of our materials are machine & human readable.
    • How: all lower case, no spaces, with dates as yyyy_mm_dd (e.g., teampsd_guiding_principles_2020_01_01).
    • Why: this naming convention is best for both human and machine readable applications.
    • Examples of major benefits we rely on on Team PSD:
      1. You can sort files by dates in a standard way (e.g., yyyy_mm_dd).
      2. You can include human readable file name in a URL, without it breaking the link (e.g., https://mtl.how/camhd_kfgc).
      3. All lowercase font does not break our primary statistical R coding language, which is case sensitive (e.g., A and a are different variables).
  • Use email for any private discussions. Host any private files in password-protected locations or in folders behind the VA firewall. When in doubt, ask an HQ member or err on the side of caution.
  • Make sure your work and accompanying documentation allows other team members or scientists in the field to reproduce and understand your work without further questions.

3.3.2 High Visibility

Our work has high visibility.

  • Keep in mind we work under the federal government of the United States for the Veterans Health Administration (VHA), the largest integrated healthcare system in the country. Everything we produce is a reflection of the VHA.
  • We work with a wide range of nationally-distributed partners both internal and external to VA, including very important and high-level stakeholders. Double check the role and responsibility of people you are communicating with before you do.
  • With these partners, we work in a participatory learning manner and iterate based on feedback from the field to ensure our work is responsive to ongoing changes.

3.3.3 Team Time

Any time saved is team time.

  • Ask questions early and often to prevent escalating issues down the road. Refer to existing resources (cheatsheets, checklists, etc.) as well for clarification.
  • Double check all work before handing it off to the next team member to reduce rework.
  • Think through dependencies across the team and partners and prioritize work based on the most recent information you have.
  • Manage workflow asynchronously (via GitHub) and only schedule meetings when absolutely necessary.

3.3.4 Communication

Use effective communication (across all types of communication including emails, GitHub, Lucid, etc.).

Assume everyone you’re communicating with is smarter than you and cares more than you and is busier than you

  • Keep it short - use an active voice.
  • Use common words - avoid acronyms and technical jargon.
  • Use bold or italics to emphasize main points.
  • Use bullet statements to structure ideas.

Include the full context and details necessary to make an informed decision

  • List the “Bottom-Line-Up-Front” - what do you want?
  • State known facts bearing on the problem - what is important for people to know?
  • List assumptions - what gaps have you identified?
  • Describe solution alternatives and make recommendations where possible.
  • Make specific “asks” to individuals using the “@” function in discussion threads and indicate when you need a response.

Maintain a complete record

  • Use previous records of Teams meetings and GitHub discussions to track decisions and stay informed.
  • Maintain conversation records in discussion threads for discussions had outside of established meetings.
  • Follow up - an action passed is not an action complete.

Stay positive

  • Use emojis.
  • Use appropriate humor.
  • Use your webcam whenever possible.

3.3.4.1 Active Listening

We use active listening skills to ensure understanding and accurate tracking.

  • We work daily with team members and partners that are experts in their respective fields, and it is easy to lose track of a complex discussion.
  • We’ve found that reflecting-back requests and decisions in your own words has been the best method to reduce miscommunication and to keep track of all of our decisions or issues accurately. If you ever need help while scribing, always “tap” someone else on the team for assistance.
  • Ask for clarification and slow down if necessary.

3.3.4.2 Active listening is a stance

Taking the stance that misunderstanding is the norm and using skills appropriate to that reality.

General skills: Reflecting content for efficiency and interpersonal rapport (i.e., avoiding rework and frustration)

  • “When in doubt, check it out” - It’s the listener’s job to help the speaker understand what they are getting or what they missed.
  • Let the speaker know when you’re falling behind. Stop them before the conversation exceeds your understanding.
  • Use the same precise language/vocabulary, esp. Team PSD or MTL terms with specific meanings.

“Chunking” components of what someone said (e.g., listen for the “and” when someone is speaking)

  • Listen for the main points.
  • This is a key skill to keep up with complex ideas in the moment (You don’t want to get bogged down on point 1 and miss point 2).

Reflection of feeling - listen for the feeling words and reflect them exactly.

  • Using the exact feeling word you heard is the safest way to ensure the person feels heard.
  • Don’t reflect “frustration” when someone is expressing “disappointment” or “stress.”
  • You can use these for non-verbals too.
  • If you hear a feeling, it’s best to address it in your reflection.

Synthesize two person’s ideas - or just two ideas (or more)

  • e.g. “on the one hand, I hear Anthony saying…” and “on the other hand, I hear Stacey saying…”
  • Synthesis brings the ideas together. It is not just active listening both ideas as separate ideas.

Summarize - used when it’s time to wrap up, and move on, and a lot of ideas have been said.

  • This is not the time to bring up new points. If you do have something to add, say it first.
  • It’s best to end with a summary of the key points of consensus and the key take aways of what to do next.

After you reflect, add to what the person is saying (a key to improv is “yes, and…” - don’t reject ideas, just try to understand first and then build on the ideas).

  • Forward momentum comes from making sure you understand the person and add your own contribution.
  • Listening is about being present to understand; you do not have to agree with what a person is saying.
  • Before you even get to agreement/disagreement, make sure you have a shared understanding first.
  • KEY IDEA: Really attend to make sure you understand. Do NOT think about your response when you should be listening.

3.3.4.3 Things to look out for when listening

Know thyself: You are the only one who knows whether it’s time to multi-task, focus, scribe/document as a form of listening, or whether typing would be a distraction.

Repetition cycle: If the speaker you’re listening to keeps repeating something, your first move should be to assume that you’re missing something and ask what it is.

Your own feelings: If you’re starting to feel frustrated, it’s a key sign that you should use active listening to get back on the same page.

3.4 Policy

3.4.1 Scope

This policy applies to all Team PSD members involved with research under grants, R21DA042198, R01DA046651, IIRI01HX002521 commonly known as Modeling to Learn (MTL). This policy is subordinate to new or existing Veterans Administration (VA) policy. Any issues identified that are contrary to VA policy or federal laws should be brought to the attention of the Principal Investigator, Lindsey Zimmerman, ().

3.4.2 Purpose

This policy details the governing principles, definitions, responsibilities and procedures for managing cards, issues, & pull requests in the Kanban production management system for the issue_tracker, feature_tracker, document_tracker, and manuscript_tracker GitHub projects for the Team PSD. Finally, the policy will describe how MTL and overarching research experimental design and reporting with be coordinated.

3.4.3 Responsibilities

  • Principal Investigator - Provides overall direction and guidance to Workflow Leads. Coordinates research activities and prioritizes activities within the MTL program with the HQ workflow
  • Co-facilitators - Gathers field information and provides feedback to Workflow Leads regarding the performance of a product in the teaching/learning environment. Facilitate Modeling to Learn 12 Session Program with identified clinics for the R01 and IIR grants.
  • Co-investigators - Oversees the science and research dependencies across the project.
  • Workflow Leads - Oversees the production and maintenance of their products in terms of quality, cost and delivery, responding to the needs of the projects as defined by co-investigators and co-facilitators.
  • HQ Proxy - An individual who is a member that supports a specific workflow and can participate in the absence of the Workflow Lead to represent the interests of a workflow. A Proxy has the decision-making authority of the Workflow Lead they represent. The Workflow Lead must still provide detailed and concise documentation of questions and scope on bugs and features related to their workflow in the Workflow Leads meeting agenda.
  1. Checks the bug_tracker, feature_tracker, document_tracker, and manuscript_tracker (as related to the workflows) daily to identify & assign interdependencies from the labels list used as a dependency check
  2. Ensures that all cards are updated in the issue_tracker, feature_tracker, document_tracker, and manuscript_tracker (as relevant to the workflow) in include a due date once they have been triaged with HQ & other workflow leads.
  3. Trains respective workflows in changes to policy & procedures for Kanban workflow.
  4. Estimates the time required to fulfill completion of bugs (issues) & features.
  5. Attends weekly Monday 8am PST / 11am EST Workflow Leads meeting or alerts the HQ Proxy (see above) in their stead, should they not be able to attend.

3.4.4 Redundancy Success Mode

Failure mode:

Email and person are in an individual member’s email, where no one else can run it down.

Success mode:

  • Information is in the manual.
  • Everyone is trained.
  • Shared resource, where others can track and run things down (Email, Team PSD folder, manual, LucidChart, ZenHub/GitHub).
  • Engineering - Redundancy

Solves scaling problems.

Platforms not people. Network effects. “Hub & Spoke” failure mode.

3.4.5 Workflow/Schedule

Team PSD is now on a flex schedule with 8 hours of individual flex time & focus blocks for each workflow.

The schedule depicts a standard Team PSD Tour of Duty from 8AM-4PM Pacific Mon-Thurs. It is designed to increase individual flex time and increase individual focus time, while decreasing “set switching” among workflows.

  • Each user workflow meeting is depicted by a box.
  • Time to prepare for the user workflow meeting or follow-up after the meeting is depicted by color.

Depicted is 16 scheduled hours of user workflow meetings per week (40% of a 40 hour week). And, no less than 8 hours of individual focus time (either through a “4 10s” or a “5 8s” schedule.

Percent of the 32 hours focused on each of these major Workflows:

  • 31.25% Cross-Cutting (Workflow Leads, Support Workflows, HQ)= 10 hours/week
  • 25% + Flex up to 50% for Research (Manuscripts, Co-I, Qual, Quant) = 8 hours/week
  • 21.8% MTL/Operations (QIICs/Facilitate, Quant, Sim/Models) = 7 hours/week
  • 20.3% Team PSD & NCPTSD Division Participation (team_temp, team_time, All Hands NCPTSD Staff meeting) = ~6.5 hours/week

The table below provides information about each of the five Team PSD value streams.

Value Stream Description
Education Activities in support of hiring, onboarding, and strategic growth, training & educational media (e.g., online resources & videos), mentoring and professional development. This includes intra-professional medical education, (e.g., CME/CEU), research training programs, and development of professional competencies in healthcare improvement.
Portfolio Activities to plan, secure, track, audit and analyze a diverse portfolio of financial resources and ensure compliance with federal, state and local, regulations and authorities, e.g., contracting, intellectual property, personnel and information security.
Research Activities in support of peer-reviewed and/or human subjects protected scientific research designed to develop generalizable knowledge, e.g., IRB modifications, DART requests, Statistical Analyses of tests of specific aims, development and submission of peer-reviewed manuscripts.
Operations Activities in support of daily healthcare operations and quality improvement (e.g., meetings w/VA healthcare operations partners to improve health service delivery).
Development Activities in support of technological development that does not include peer reviewed human subjects research or VA healthcare operations (e.g., development of a technology or media that does not directly support peer reviewed human subjects research and developed with minimal coordination w/VHA Operations).

3.4.6 Request Time Off

1. Submit your official request through VATAS or ADP. Submit your leave in the VATAS or ADP system.

2. Document approval with an email chain approving leave (whether VA or PAVIR). Please submit an email to me (VA employees cc Randy and Karen) about the leave you would like to take and I will approve it.

3. Prepare monthly epics with leave in mind. We do prefer to have a plan for coverage across the team, and plan our sprints accordingly.

4. Add to mtl.info and mtl.help calendars. Please add your OOO to shared calendars.

5. Use an OOO automatic email whenever you’re OOO during standard VA Tour Pacific Time (Mon-Fri 8AM-4:30PM). This helps VA partners to see that you’re out across VA/Microsoft platforms.

3.4.7 GitHub Labels Table

Labels shall be assigned a color, expressed in lowercase and use an underscore in lieu of a space. Below is the list of labels and their purpose:

Label Purpose
document expresses a dependency for at least 1 of 5 levels (describe, detail, document, disseminate, depend) of documentation to be tracked on the document_tracker. The point person for each level of documentation will be responsible for checking the issue & feature tracker for dependencies (describe: Jane & Debbie; detail: Tom & Lindsey; document: Stacey & HQ; disseminate: Lindsey, Anthony; depend: Lindsey, Jane, & Jessilyn)
facilitate alert all of the facilitate workflow to an issue that may affect facilitation (Jane, Debbie, Jenn, Lindsey, Stacey)
hq alert to HQ workflow (Lindsey, Stacey, Jennifer, Anthony as Quant representative) to track
manuscript expresses a dependency on manuscripts to be tracked on the manuscript_tracker
model alert to Tom & James of a dependency on the model workflow
pi alert to PI/Lindsey that an action or decision is necessary
qual alert to David, Kathryn, Swap, Lindsey, Jessilyn, & Stacey to of dependency on the qual workflow
quant alert to Anthony & Ash of dependency on the quant workflow
sim_ui alert to sim_ui workflow lead of an issue or dependency that affects the sim_ui. The workflow lead will evaluate sim_ui impacts and collaborate with other workflow leads to determine an adequate resolution.
urgent indicates a short amount of time is available for resolution and needs to be prioritized by workflow
ees description to come
vapor description to come

3.5 Project Management [flowmap needed]

  • NOTE: Team PSD Scientific Values guide how and why we synthesize the approaches below

3.5.1 Waterfall

Team PSD integrates Waterfall principles because:

  • We are a research team that cannot deviate in quality, scope, timeline or cost from our thoroughly developed and vetted research plan.
  • Therefore, there are several well-defined scientific requirements where change cannot occur - unless the PI and co-I senior scientists advise an operational definition or implementation detail is within the scope of the federally funded and registered, scientifically peer-reviewed, and VA ORD and Stanford IRB reviewed procedures.

3.5.2 Agile

Team PSD integrates Agile principles because:

  • Our research team’s method is a an implementation of participatory system dynamics in a national production environment.
  • These methods historically have iterated in-person locally.
  • Our innovation is local priorities, local data and local insights at national scale, made possible using online platforms and virtual processes.

Team PSD integrates Waterfall and Agile principles because:

  • We are using a funded a priori research design (Waterfall) to study an implementation science (Agile) problem.
  • Producing generalizable knowledge requires implementation scientists to follow and report publicly progress completing a rigorous, replicable plan (Waterfall).
  • However, the context is not a tightly controlled laboratory, rather we are working with operational partners to implement our innovation the dynamic context of real world healthcare operations (Agile).

3.5.3 Lean

Team PSD integrates Lean principles because:

  • Team PSD processes are assessed for continuous improvement (kaizen/muda elimination).
  • Team PSD communication is designed to reduce rework (muda/waste).
  • Team PSD encourages minimum viable products to find and synthesize team and partner expertise (just-in-time).
  • Team PSD platforms are designed to increase visual production controls (kanban/signboard; andon alerts).
  • Note: VA uses Lean as a primary approach to healthcare quality improvement.

3.5.3.1 Andon Alerts

  • Team PSD uses Andon Principles to manage bug reports and triage the issue as soon as possible.
  • Team PSD follows three core ideas when addressing an andon.
    • Scope - Analyze the impact of the issue and determine who the critical users are that need to be informed.
    • Alert - As soon as scoping is complete inform the impacted users of the issue and how to track it’s resolution. Providing work-arounds if needed.
    • Communicate - Until the issue is resolved maintain open lines of communication and provide consistent status updates.

3.5.4 Scrum

Team PSD integrates Scrum principles because:

  • We need to be able keep work produced by our transdisciplinary research team on the same page with our cross-functional team.
  • Therefore, we use sprints (epics/milestones)so that the team priorities can be aligned/re-aligned efficiently.
  • We also use workflow leads (scrum masters), workflows (owners) and workflow meetings to benefit from the efficiency of divvying up/delegating, while also identifying dependencies and remove blocks.
  • We use GitHub/ZenHub and daily huddles to assign, scope, prioritize, manage and review our capacity, requirements, estimates - this includes Project Kanbans, Issue Cards, Pipelines & Reporting.

3.5.5 Team Pain Points

Team PSD Pain points we are working to better address:

  • Need tight collaboration without unnecessary overhead, so culture of participation is maintained
  • Need rapid iterations that prototype, check early, and synthesize inputfrom key stakeholders as efficiently as possible, so we aren’t pulled in wasteful directions or adding unnecessary rework
  • Need a standard set of principles and processes, so our production platforms are stable at scale.

3.5.6 Continuous Collaboration

Continuous Collaborative Iteration Cycles (e.g., “DevOps”)

  • Standard Operations > Workflow Meetings > Monthly Priorities > Requirements gathering standardization for features
  • Tools for team members to understand cross-functional “stakes” efficiently
  • “Continuous integration” to avoid “merge conflicts” - Quotes are meant to clarify that we have this problem at a communication and conceptual level, not just a code level.

We need the code level next…because…

  • Ideally…testing, deploying and documenting would be increasingly automated.
  • REALLY need continuous documentation and training.

3.6 Project Trackers

Team PSD uses the following 4 project trackers on GitHub to track incoming bugs, develop new features, create/update documentation, and ensure the progression of submitting manuscripts, all from a glance by looking at each of their respective trackers as a dashboard to see the status, progression, and completion of open issues.

3.6.1 bug_tracker

The bug_tracker is divided into 6 columns described below. The purpose of the bug_tracker is to provide triage and track the disposition of issues that require action by one or more workflows. Issues labeled as “bugs” will be tracked here.

3.6.1.1 Types of Bugs

3 kinds of “Bugs” - Bugs are rapport-builders and need to be handled in relationship w/our partners.

  1. “Bug” - something not working as intended.
  2. “Andon” - “Stop the Line” - the problem could lead to a reduction in the quality of MTL

Scope - Impact by time Alert - Impacted partners Communicate - until resolved.

  1. “Problem” - Needs a development plan because the bug keeps popping up.

3.6.1.2 Bug Tracker Values:

  • Reviewed daily morning & afternoon for empathetic team practice (Empathic concern for our VA partners: patients, providers, policymakers).
  • User empathy - We want to solve user pain points.
  • Participatory - so being responsive to our users/partners is foundation to our team.
  1. needs_triage - This column is the main intake for all issues assigned to the issue_tracker. All issues requiring a disposition will land here. When an issue lands in this section, any team lead may review it and alert other leads as to the action required.
  2. validated_actions (ranked) - This column contains all issues that have received a “bug” disposition by a Workflow Lead(s) and has been provided sufficient details to fix the problem. This is a rank-ordered list based on due dates indicated in the title. The rank-order of this section can be changed at any time. If an issue is determined to be a “feature,” it will be placed in the feature_kanban (see feature_kanban below).
  3. under_development - Bugs that are currently under development are listed in this column. This section may not be reprioritized, and contents are addressed first-in-first-out by respective workflows.
  4. testing - Bugs that are currently being tested are listed in this column. Some issues may skip this step and go directly from under_development to done.
  5. done - This column contains completed issues. Responsible Workflow Leads shall communicate completion of the action to the originator in the issue thread. The originator shall review the action, indicate their satisfaction/dissatisfaction. Once the originator is satisfied, the originator should close the issue and it will automatically go in the closed column.
  6. closed - This column contains issues closed by the originator from the done column. Any issue can be reopened as necessary.

3.6.2 feature_tracker

The feature_tracker is divided into 9 columns described below. The purpose of the feature_tracker is to report and maintain information regarding the analysis of dependencies and work-content, and track the progress of issues that require development. Issues labeled as “features” will be tracked here.

  1. work_breakdown - Validated feature requirements that have received a disposition are listed here. Issues in this column are analyzed by Workflow Leads for dependencies, effort content and milestones they may support (see Appendix 1 - issue_template). Issues will progress from this section to either the operations_to_do (ranked) or the research_to_do (ranked) sections.
  2. operations_to_do (ranked) - Features that require research, design and development of products in one or more workflows are listed here in order of priority by due date indicated in the title. Issues here may be reprioritized at any time.
  3. research_to_do (ranked) - Features that support research, evaluation and documentation efforts directly related to supporting grants or higher-headquarters reporting requirements are listed there in order of priority by due date indicated in the title. Issues here may be reprioritized at any time.
  4. under_development - Operations or research features under development are listed here. This section may not be reprioritized, and contents are addressed first-in-first-out by respective workflows.
  5. functional_testing - Features that are currently being tested for basic functionality (i.e. does it work?) are listed here. Some issues may skip this step and go directly from “under_development” to “done.”
  6. done - This column contains completed issues. Responsible Workflow Leads shall communicate completion of the action to the originator in the issue thread. The originator shall review the action, indicate their satisfaction/dissatisfaction. Once the originator is satisfied, the originator should close the issue and it will automatically go in the closed column.
  7. closed - This column contains issues closed by the originator from the done column. Any issue can be reopened as necessary.
  8. future_release – This column contains unresolved feature ideas that would be great to include in a future MTL release, but currently is not a pressing need.

3.6.3 document_tracker

The document_tracker is divided into 5 columns described below. The purpose of the document_tracker is to track documentation dependencies at 5 levels for each of the 12 sessions of Modeling to Learn. There will be a card for each session of the 12 sessions in each of the 5 Kanban columns which will be closed & reopened as interdependencies get identified.

Each column also has a “meta” card which is used to indicate interdependencies that are relevant to all/most of the 12 sessions as well as policy & workflow decisions specific to that documentation column.

  1. describe_learners – Documentation dependencies relevant to learners, including SEE guides
  2. detail_facilitators - Documentation dependencies relevant to facilitators, including SAY guides & one-pagers
  3. document_team - Documentation dependencies relevant to Team PSD, including cheatsheets
  4. disseminate_scientists_va - Documentation dependencies relevant to co-investigators & larger scientific audience, including progress reports, code, grants, etc.
  5. depend_products - Documentation dependencies relevant to other MTL products, including videos

3.6.4 manuscript_tracker

The manuscript_tracker is divided into 10 columns described below. The purpose of the manuscript_tracker is to track progress of manuscripts through 10 major stages until ready for publication.

DO NOT post any direct manuscript content to GitHub; all drafts and related materials will be posted on the relevant OSF project. There will be a card per manuscript that moves through the tracker.

  1. 01_osf_project – OSF project is created for this manuscript with all relevant materials posted, Zotero & GitHub integrations approved, cheatsheets linked, and relevant people added
  2. 02_authorship – Initial authorship weights and task division are identified
  3. 03_write_analyze – Manuscript is written and materials for analysis are produced.
  4. 04_edit – Manuscript goes under rounds of editing and revision between writing team.
  5. 05_approve_letter – Team working on manuscript gives final approval and drafts letter to editor
  6. 06_submit_under_review – Manuscript is submitted to relevant journals and is undergoing review
  7. 07_revise_and_respond – Manuscript feedback is received from journals and team working on manuscript revises it accordingly
  8. 08_resubmit – Manuscript is resubmitted to relevant journals
  9. 09_accept – Manuscript is accepted for publishing by journals
  10. 10_publish_publicize – Manuscript is published!

3.6.5 research_bug_tracker

The research_bug_tracker is divided into 6 columns described below, similar to the bug_tracker. The purpose of the research_bug_tracker is to track bugs related to Team PSD research.

  1. needs_triage - This column is the main intake for all issues assigned to the research_bug_tracker. All issues requiring a disposition will land here. When an issue lands in this section, any team lead may review it and alert other leads as to the action required.
  2. validated_actions (ranked) - This column contains all issues that have received a “bug” disposition by a Workflow Lead(s) and has been provided sufficient details to fix the problem. This is a rank-ordered list based on due dates indicated in the title. The rank-order of this section can be changed at any time.
  3. under_development - Bugs that are currently under development are listed in this column. This section may not be reprioritized, and contents are addressed first-in-first-out by respective workflows.
  4. testing - Bugs that are currently being tested are listed in this column. Some issues may skip this step and go directly from under_development to done.
  5. done - This column contains completed issues. Responsible Workflow Leads shall communicate completion of the action to the originator in the issue thread. The originator shall review the action, indicate their satisfaction/dissatisfaction. Once the originator is satisfied, the originator should close the issue and it will automatically go in the closed column.
  6. closed - This column contains issues closed by the originator from the done column. Any issue can be reopened as necessary.

3.6.6 research_feature_tracker

The research_feature_tracker is divided into 6 columns described below. The purpose of the research_feature_tracker is to track features that support research, evaluation and documentation efforts directly related to supporting grants or higher-headquarters reporting requirements.

  1. work_breakdown - Validated feature requirements that have received a disposition are listed here. Issues in this column are analyzed by Workflow Leads for dependencies, effort content and milestones they may support (see Appendix 1 - issue_template).
  2. to_do (ranked) - Features that support research, evaluation and documentation efforts directly related to supporting grants or higher-headquarters reporting requirements are listed there in order of priority by due date indicated in the title. Issues here may be reprioritized at any time.
  3. under_development - features under development are listed here. This section may not be reprioritized, and contents are addressed first-in-first-out by respective workflows.
  4. functional_testing - Features that are currently being tested for basic functionality (i.e. does it work?) are listed here. Some issues may skip this step and go directly from “under_development” to “done.”
  5. done - This column contains completed issues. Responsible Workflow Leads shall communicate completion of the action to the originator in the issue thread. The originator shall review the action, indicate their satisfaction/dissatisfaction. Once the originator is satisfied, the originator should close the issue and it will automatically go in the closed column.
  6. closed - This column contains issues closed by the originator from the done column. Any issue can be reopened as necessary.

3.7 Shared Outlook Inboxes

3.7.1 MTL Help

email and calendar is used for any Team PSD operations workflows, including communication with the clinic teams using the Modeling to Learn quality improvement program.

3.7.2 MTL Info

email and calendar is used for any Team PSD research workflows.

3.7.3 How to Get Access

If you are unable to currently access either of the two inboxes:

  1. Create an IT ticket at yourit.va.gov requesting access to the respective inbox.

  2. Email your ticket number and request to Frank Behrens (search in VA GAL), who will pick up the ticket from the queue and complete it. (***Note: this cannot typically be completed by your local IT, and must be assigned to Frank).

3.8 Quality Assurance Cadence

  • Value Streams: Education, Portfolio, Research, Operations, Development
  • Workstream:
  • Workflows: All
  • Video

Teams Channel Link

3.8.1 DEV Cadence QA

Dev is responsible for completing user-hypothesis testing, delineating MVPs, verifying user stories, thinking through the requirements and impact, and deciding (or providing recommendations) on if/how we should take the first step.

3.8.2 TEST/QA Cadence

Test is responsible for seamless integration by making sure the resource and all of its potential impacts are thought through and updated in a manner consistent to existing workflows, language, facilitation principles, etc. The test person should be identified by role and expertise in workflow.

3.8.3 PROD QA

Prod is responsible for the final check and polish before official release and branding to all users.

3.8.4 The QA Process

The Dev person will manage and create a series of three cards. At each step, the QA assignee will sign off on the GH issue card before handing it off to the next step. - Dev: Same as original issue card that that Dev person will update with understanding of key requirements, wireframes, etc. Includes checklist from master document card of all the existing features + newest feature/bug fix that the Dev person will test. - Test: Dev will include checklists from master document card of all existing + new features/fixes that need to be tested, as well as any testing scripts as needed. Test will document all dependency checks thought through and updated to ensure seamless integration. - Prod: Dev will include any testing scripts for new feature/fix. Test will include brief overview of checks & updates completed to ensure seamless integration.

3.9 Story Points

Team PSD uses the modified Fibonacci sequence (1, 2, 3, 5, 8, 13, 21, etc.) for exponential scaling to account for complexity (interdependence), effort, and repetition. They are a relative measure rather than an absolute one, which allows teams to compare the effort required for different tasks without getting stuck in specifics like units of time.
Below is a standard scale for story points related to each value stream.

Education

Story Points Description
1 Complete a small task related to mentorship/education.
2 Review work for mentees or learners; facilitate a meeting.
3 Write curricula, plans, syllabi, and/or research/organize materials.
5 Coordinate with programs to review or select proposals/candidates.
8 Participate in a major training event; lead a workshop.
13 Complete a Team PSD train cadence (wk1, wk2, wk3, wk4).
21 Launch a large training program.

Portfolio

Story Points Description
1 Email a portfolio partner.
2 Update an official document, record, or platform (includes #adjudicate_memo).
3 Coordinate regular communications & audit with portfolio partners.
5 Produce a report for key stakeholders.
8 Plan or develop a new budget for a grant or contract.
13 Monitor, analyze and coordinate salaries, contracts and purchasing (wk1, wk2, wk3, wk4).
21 Finalize a contract.

Research

Story Points Description
1 Complete a small task that moves research forward.
2 Review work for integration/QA.
3 Write analyses, visualization (table/figure), etc.
5 Coordinate co-Is in the incorporation of methods to test specific aims; submit a conference abstract.
8 Complete a large set of interrelated analyses; participate in Grant Review panel.
13 Sync the draft, edit, revise, and publish manuscript cadences (wk1, wk2, wk3, wk4).
21 Submit a monthly epic grant pre-award deliverables.

Operations

Story Points Description
1 Complete a small task that moves healthcare operations forward.
2 Coordinate work with operations workflow users.
3 Advance a major milestone/dependency impacting progress across users.
5 Coordinate several interrelated workflows to ensure a new feature is developed.
8 Complete a large set of tasks testing and documenting a PeopleOps workflow.
13 Complete a Team PSD operations cadence (wk1, wk2, wk3, wk4).
21 Launch a release: e.g., MTL 3.0, MTL for all.

Development

Story Points Description
1 Check a specific functional requirement.
2 Test a specific functional requirement.
3 Document the new functional requirement across users.
5 Train multiple users in use of the new functional requirement.
8 Address all related dependencies related to the new functional in manuscripts & videos.
13 Complete a Team PSD development cadence (wk1, wk2, wk3, wk4).
21 Launch a release.