- IssueType: Epic
  Project: LKQ
  Summary: Improve SQUADs linux-log-parser
  Epic Name: Improve SQUADs linux-log-parser
  Description: |-
    Today’s linux-log-parser in SQUAD can be improved by how it matches the
    logs. Sometimes it matches too much and too little.

    To have a SQUAD linux-log-parser that classifies logs correctly is
    essential for regbot.

    With this rework of the log parser the log parser failures will be better
    matched, that means more descriptive names of the failures, the log line
    will only map to one failure.


    Deliverables
    * Use the same patterns as LAVA has
    * have more descriptive test failure names
    * Have a new API to search for strings in the logs

  AssigneeEmail: anders.roxell@linaro.org
  Due date: 2023-06-30
  Share Visibility: [charles.oliveira@linaro.org, ben.copeland@linaro.org]

- IssueType: Story
  Project: LKQ
  Summary: "Use the same patterns as LAVA’s has"
  Description: |-
    Update SQUAD's log parser to classify the same issues LAVA classifies.

    'https://docs.google.com/document/d/1acRdyfT_df_gM83a_xNRSky-PBRlgj5Z8U7OeDCwiY0/edit#heading=h.yj0js9vjo69d'

  AssigneeEmail: charles.oliveira@linaro.org
  EpicLink: "Improve SQUADs linux-log-parser"
  OriginalEstimate: 2d
  Share Visibility: [anders.roxell@linaro.org, ben.copeland@linaro.org]

- IssueType: Story
  Project: LKQ
  Summary: "Have more descriptive test failure names"
  Description: |-
    '
    If a regular expression is added to the KERNEL_MESSAGES “kind” to find the
    name of a “WARNING”, “BUG: KASAN”, “Internal error: Oops:”, “BUG: KFENCE”,
    “kernel BUG at” or “BUG:”... the test failure name can be more descriptive.

    Add examples what to search for in the different multiline logs

    Example 1:
    “BUG: KASAN: slab-out-of-bounds in kmalloc_oob_right+0x3d4/0x418”
    Should be “kasan-slab-out-of-bounds-in-kmalloc_oob_right-<sha>”, the sha
    should be generated as we have today.

    Example 2: inside ‘-[ cut here ]-’
    “WARNING: CPU: 0 PID: 1 at lib/refcount.c:25 kthread_stop+0x238/0x3b8”
    Should be “warning-lib/refcount.c_kthread_stop-<sha>”

    Example 3: inside ‘- cut here -’
    “PC is at map_id_range_down+0x74/0”
    “map_id_range_down-<sha>”

    https://tuxapi.tuxsuite.com/v1/groups/linaro/projects/lkft/tests/2cwYIyaw7HqyEtSdGn2QMJCcmwJ/logs?format=html

    Example 4: kfence inside multiline start/end ’========’
    "BUG: KFENCE: out-of-bounds read in test_out_of_bounds_read+0x1a9/0x390"
    Should be "kfence-out-of-bounds-read-in-test_out_of_bounds_read-<sha>",
    the sha should be generated as we have today.

    https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20240402/testrun/23267594/suite/log-parser-boot/test/check-kernel-kfence-b95df134389ef2deb4e19f03de7f87346bba9163b5b10da58ebb93a2383961e8/log
    '


  AssigneeEmail: charles.oliveira@linaro.org
  EpicLink: "Improve SQUADs linux-log-parser"
  OriginalEstimate: 5d
  Share Visibility: [anders.roxell@linaro.org, ben.copeland@linaro.org]

- IssueType: Story
  Project: LKQ
  Summary: "Have a new API to search for strings in the logs"
  Description: |-
    '
    LKFT produces a lot of logs that gets parsed with the log parser and relies
    heavily on the log parser. LKFT have the need to add its own patterns to
    match on when new issues pops up. Meaning, if a SQUAD build_id/testrun have
    a specific issue in the logfile it can be submitted via a new API. The
    return would be the new test_id.

    Create a new API endpoint in squad to be able to do a log search after a
    pattern, like ‘CPU1: failed in unknown state’

    An example of a warning that easily can get missed.
    https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20240219/testrun/22731340/suite/boot/test/gcc-13-lkftconfig-64k_page_size/log

    From the admin interface the admin user should be able to remove a user
    defined linux-log-parser string.

    How this new API can look like:
    /api/projects/<id>/add_log_search=‘CPU1: failed in unknown state’&number_of_builds_to_look_backwards=10
    /api/projects/<id>/add_log_search_start=‘CPU1: failed in unknown state’&log_search_end=’asdflj’&log_search_kind=’bad-things’
    '

  AssigneeEmail: charles.oliveira@linaro.org
  EpicLink: "Improve SQUADs linux-log-parser"
  OriginalEstimate: 3d
  Share Visibility: [anders.roxell@linaro.org, ben.copeland@linaro.org]

- IssueType: Story
  Project: LKQ
  Summary: "regbot: create custom log parser patterns"
  Description: |-
    Make it possible to create custom log parser patterns from regbot into multiple projects.
    Form data
    * projects list (ids)
    * pattern
    * number of builds to look backwards
    Listen to a callback from SQUAD when the new failures are ready.

  AssigneeEmail: katie.worton@linaro.org
  EpicLink: "Improve SQUADs linux-log-parser"
  OriginalEstimate: 3d
  Share Visibility: [charles.oliveira@linaro.org, anders.roxell@linaro.org, ben.copeland@linaro.org]

- IssueType: Story
  Project: LKQ
  Summary: "background create new failures from the new search string"
  Description: |-
    When a new log parser pattern are submitted, SQUAD should create a
    background task that runs over all logs in the projects that have this
    plugin enabled. From the newest build and back 200 build_id's back from the
    latest seen failure in the project.
    All the new tests

  AssigneeEmail: charles.oliveira@linaro.org
  EpicLink: "Improve SQUADs linux-log-parser"
  OriginalEstimate: 4d
  Share Visibility: [anders.roxell@linaro.org, ben.copeland@linaro.org]

