View Source

For an archive of meeting notes pre-dating the Aether Project moving from ONF to LF, click here.

Recordings of TST meetings are posted on the Aether YouTube Channel: https://www.youtube.com/@Aether5GProject

If you need technical help from the Linux Foundation IT staff, please submit a ticket (typically requesting a Project Service). When creating a ticket:

Component = 'Release Engineering'
Product = Aether

3-18-25 (Topics for Next Meeting)

...

3-11-25 (Today's Agenda)

Infrastructure
- GitHub Actions
- AWS Images
Mixed Licenses
- Apache 2.0 uses more restrictive language than MIT, but they are roughly equivalent
- Document in README (probably sufficient at the "GitHub organization README" level)
SD-Core
- Currently addressing issues related to new golang dependency versions
  - Newer golang dependencies require semver format for the go module (i.e., go 1.23.0 instead of go 1.23)
- Significant progress on 3GPP release upgrade (release 18): All NFs are now with release 18 for the SBI
  - Several issues to address such as callbacks, replace some structs that were removed in newer 3GPP release, among others
SD-RAN
- Open Source Near-RT RIC Comparison Report (by Rimedo Labs) - see today's blog post
  - Planning a weekly blog highlighting SD-RAN features
  - Culminates in a webinar to bring it all together (May 27th) - register for webinar
- SD-RAN v1.5 Techinar Video
- srsRAN has been tested with other RICs, but...
  - SD-RAN sends an "E2 Connection Update" message that srsRAN does not respond to
  - SD-RAN may need to be robust when there is no reply
OnRamp
- Blueprint for "Multi-Hop Access" network released
- A student working on service monitoring; just opened a PR
  - General idea: eliminate per-NF monitoring; always go through metric_func

3-4-25

Infrastructure
- GitHub Actions
  - Pushed an initial version last night
  - https://github.com/onosproject/.github/pull/7
- AWS Images
  - New 22.04 works
  - New 24.04 fails in Docker install
  - See aether-packer repo: https://github.com/opennetworkinglab/aether-packer
  - See PR for aether-jenkins: https://github.com/opennetworkinglab/aether-jenkins/pull/16
SD-Core
- New Helm Charts were released (version 2.2.2)
  - Fixes (avoids) 100% CPU utilization
- 3GPP Release upgrade (release 18)
  - SMF: Addressing issue with multipart response message
  - NRF: Addressing missing callback
- How to deal with multiple licenses; e.g., MIT & Apache (ask for LF advise)
SD-RAN
- SD-RAN v1.5 Techinar is tomorrow, March 5th at 8:00am Pacific. Register to attend
OnRamp
- Reviewing latest PR with 2.2.2 upgrade.
- Discussion about scope: Ease learning curve or industrial strength deployment robustness
- Testing Infrastructure assumes OnRamp itself is relatively static
ARA
- Discussion with Serdar Vural from Canonical about Charmed Aether SD-Core

2-25-25

Infrastructure
- GitHub Actions
  - Eric Update: ROC release and reusable workflows: After speaking with one of my colleagues last week who does a lot of work on our reusable workflows, I rearchitected the helm release to have most functionality outside of the workflow, where pretty much any action that needs to be taken is called from a small, discreet "micro-action". The result is workflows that are far cleaner, and having each bit of work in its own action allows for much clearer documentation. Even if it's still at a WIP point, I'll push my code later so folks can get a look at it and provide feedback.
- AWS Images
  - will create a new aether-packer repo for VM spec
  - plan is to support 22.04 and 24.04; likely to deprecate 20.04
  - Eric Update: I've got code-based, reproduceable builds of Ubuntu images for Jenkins up and running. I'm currently testing the produced AMIs with this job: https://jenkins.aetherproject.org/view/all/job/Aether_OnRamp_Manual-test/. It looks like the most recent attempt failed due to a missing key. That may actually be something that needs to be changed in the job, since the best practice would be to have the key stored on the Jenkins server rather than embedded in the image.
SD-Core
- New Helm Charts are released (version 2.2.1)
  - Remove cert-manager and use of self-signed secrets
- Issue with "stagnant" dependencies
  - golang.org/x/crypto from 0.33.0 to 0.35.0: https://github.com/omec-project/webconsole/pull/313
  - https://pkg.go.dev/gopkg.in/yaml.v2: Published: Nov 17, 2020
  - Discussion about MIT vs Apache license; will ask for LF's input (llp action item)
- Webconsole and 4G support
  - Do we need to be backward compatible with 4G? → We have decided to not support 4G, but looking for Ajay's input
SD-RAN
- Webinar (announcing v1.5) planned for March 5th.
- Will finalize v1.6 planning after the webinar
- Hardware documentation is being updated
- Team won best paper award at a Korean conference (will update Research page)
OnRamp
- srsRAN blueprint has regressed
- Currently, OnRamp only supports physical/external gNBs connected to the same subnet as 5GC host machine
  - This will cause issues on how to reach the UPF's N3 interface: This issue is present in ARA and other deployments
  - PRs https://github.com/opennetworkinglab/aether-5gc/pull/36, https://github.com/opennetworkinglab/aether-srsran/pull/10 address this limitation.
- Need to downgrade K8s (RKE) by one minor version due to issues finding compatible Helm Charts for the monitoring component
- There is a "NetPlan" issue (see Slack)
ARA
- No update.
...

2-18-25

LF "Core" Workshop
- Interest in gNBsim as a core-neutral testing tool → Easy adoption
- Interest in shared tooling for generating 3GPP APIs
- Interest in shared support for physical UEs (e.g. burning SIM cards)
- Comparison Cheatsheet → Ease of getting started (Aether strength)
- See notes: https://lf-networking.atlassian.net/wiki/spaces/LN/pages/332726273/Open+Source+Packet+Core+Workshop
Infrastructure
- GitHub Actions
  - Issues with golanci-lint: Had to close dependabot-opened PRs
- AWS Image Instances
- Enable private security reports
SD-RAN
- SD-RAN 1.5 webinar planned for March 5th, 8am Pacific
- Added the comparative analysis to Wiki (will mention on Slack too)
- Started demo recording for webinar.
- Started investigating how we can merge @Gabriel Arrobo’s PRs for 5G RSM use-case.
- Started 1.6 planning - especially for the new RAN-Simulator and API/go version upgrade.
SD-Core
- Fixed AMF issue; Helm chart update pending (multiple fixes to include)
- Looking at adding a CERT manager sub-chart
- Review security-related issues/PRs opened PSU team (Gabriel)
- Need to prune https://docs.sd-core.aetherproject.org of obsolete/deprecated info
  - e.g., references to NG40, 4G support, etc.
- Continue having progress with release upgrade:
  - Currently testing changes with SMF
  - Still addressing some "compatibility" issues
OnRamp
- Updated to RKE v1.25.16+rke2r2 and Helm v3.17.0 ( Thanks Gabriel! )
  - Remove the need of installing kubectl from source
- srsRAN blueprint seems to have regressed
- Started email exchange with Jordan & Sean (Atomix & ROC)
  - Due to loading CRDs; need to disaggregate
- Will likely deprecate support for 20.04 once we are fully supporting 24.04. → to discuss
ARA Deployments
- SD-RAN
  - Working on the support of K8s
...

2-11-25

Infrastructure
- Upgrade Jenkins plugins (Done)
- GitHub Actions
- AMIs for Jenkins → Priority for 24.04
- Enable private security reports for all active repos (see notes 2/4/25 meeting notes) → mimic what 5gc repos do on omec-project
SD-Core
- Disable inexistent Jenkins jobs (FOSSA, License/REUSE, onf/cla) from EPC/4G-related repos? [Gabriel
  - There are no active Jenkins jobs related to fossa, license, cla
- Has the time come to archive 4gc → Raise the issue with the community. /Action Item/
- Started testing SMF (3GPP release 18) with OnRamp
  - For now, facing a TLS: certificate verification issue. Currently trying to debug this issue
- Review and merged ~30 PRs related to dependency update
SD-RAN
- Release v1.5 notes are final (double check wiki)
- RIC evaluation report to be released this week (LF involved in publishing report)
OnRamp
- Multi-UPF integration test now runs
  - The issue was because OnRamp was using an outdated version of the UPF Helm Charts (correct version is v1.2.0)
- For the record:
- ROC has been problematic (probably a timing error of some kind)
  - Pain: Atomix is installed in kube-system name space so have to re-install Kubernetes
- Been updating blueprints to work with 24.04, but not rigorously (waiting for AMI – see above)
  - amp, 5gc, oai, srsran and gnbsim are updated and working with Ubuntu 24.04
ARA
- SD-RAN
  - Working on enable SD-RAN on ARA portal
...

2-4-25

Infrastructure
- GitHub Actions
- AMIs for Jenkins
Discuss recent reports
- RIC Evaluation: https://drive.google.com/file/d/13Lyqf0l91FEi-9HB7E99-u_aIbJVmYB7/view?usp=drive_link
  - What can the Aether project do better to support its nRT-RIC → dependabot (security alerts)
  - Sufficient developer community / user community
  - Table further discussion until Woojoong is available
- Security Report (with respect to SD-CORE): https://nathanielbennett.com/publications/ransacked.pdf
  - Almost entirely focused on 4G → We do not maintain 4G, and need to be clear about this.
  - Action Items:
    - Reflect this position on WIki, docs, website
    - Invite the researcher to a future TST call
    - Enable private security reports: https://docs.github.com/en/code-security/security-advisories/working-with-repository-security-advisories/configuring-private-vulnerability-reporting-for-a-repository (done for some repos; need to make sure on all active repos)
      - Done for all 5G-related repos of SD-Core.
OnRamp
- Why ROC's Atomix is deployed in the core Kubernetes namespace (kube-system)? [Gabriel]

Home > Agenda & Meeting Notes > image2025-2-3_15-2-39.png

- Is it possible to visualize metrics/measurements in the ROC? Aether documentation indicates that it is possible: https://docs.aetherproject.org/master/operations/monitor.html → Worked at one time, but not maintained. Consider it deprecated.
SD-Core
- Will add some settings in the sdcore-5g-values.yaml files related to static IP assignment, security "order" and IMS-related setting [Gabriel
  - Does it make sense to add a Pixel7-specific values file?
- Address an issue with AMF crash [Ajay
- SMF not assigning IP addresses to UEs connecting to 2nd UPF (multi-UPF blueprint not working)
- There are configurations specific to SD-Core baked into gnbsim, making it difficult to test other cores.
SD-RAN
- ...
ARA
- SD-RAN
  - Preparing SD-RAN ARA blueprint
  - Working on enable SD-RAN on ARA portal
- SD-Core
  - Refining the documentation for the Ericsson-SD-Core integration
...

1-28-25

Infrastructure
- GitHub Actions
- Generate AMI instances
SD-Core
- Webconsole changes by Canonical
  - Extending API to support adding UPFs/Slices
  - Should not have impact on legacy API
  - Includes swagger support
- Article on security flaws in open source core:
  - https://www.securityweek.com/lte-5g-vulnerabilities-could-cut-entire-cities-from-cellular-connectivity/
  - PDF of academic report: https://nathanielbennett.com/publications/ransacked.pdf
  - To revisit next week
SD-RAN
- Draft evaluation report: https://drive.google.com/file/d/13Lyqf0l91FEi-9HB7E99-u_aIbJVmYB7/view?usp=drive_link
- Let's review and plan to discuss next week
  - anything we disagree with? If so - raise it now before this is finalized
  - what do we do this this now? Anything actionable?
  - Press/PR opportunity? Can we tease out some positives to publicize?
- Reviewing an announcement of v1.5
- Working on webinar; to include a demo
OnRamp
- Jenkins now stable enough to spot regressions
- Multi-UDP Test has been failing since Jan 2nd
  - Possibly related to SD-Core chart upgrade from 2.0.0 to 2.1.0?
  - See for example https://jenkins.aetherproject.org/job/AetherOnRamp_UPF_22.04/102/
ARA Deployments
- SD-RAN
  - Preparing SD-RAN ARA blueprint
  - Working on K8S support on ARA portal
    - Take note of RKE2 dependency
- SD-Core
  - SD-Core + Ericsson RAN blueprint review
Other Topics
- ...

1-21-25

Infrastructure
- Generate new AMI instances for Jenkins (22.04 and 24.04)
- Consolidate GitHub Actions
- Complete GitHub Actions for ROC
SD-Core
- ...
SD-RAN
- ...
OnRamp
- Updated for srsRAN and UERANSIM
  - Need to update OAI blueprint
ARA
- SD-RAN
  - Prepare SD-RAN ARA blueprint
Monogoto Introduction (https://monogoto.io/)
- See https://docs.motogoto.io for documentation (including API)
- Anticipating Private 5G uptick, but focused on roaming now
- Strong emphasis on self-service

1-14-25

Monogoto Introduction (https://monogoto.io/)
Infrastructure
- Jenkins updated to recent LTS
- Aether Registry includes a proxy for Docker Hub
- Also a daily script that copies latest images from Docker Hub to Registry
SD-Core
- Enforcing MongoDB ReplicatSet or Sharded Cluster for transaction support (Ghislain)
  - Standalone mode doesn't support transactions
  - Should just work
- Support AF_PACKET mode in helm charts to enable E2E tests in GH actions (Ghislain)
  - Working towards e2e tests for PRs
  - May run into "GH runners" limitations and need Jenkins involvement
  - If this proves to be the case, existing Jenkins "Quickstart" job is close (e2e gnbsim+sdcore)
    - would need to modify to work on candidate images
- Review, test, and merge ~20 PRs opened by Dependabot (dependencies)
- Adjusting log level to reduce performance penalty
SD-RAN
- Reviewing release notes
- Will announce release
- Then will work on webinar
- Looking to August for 1.6 release
OnRamp
- SD-Core playbooks updated to use registry.aetherproject.org/proxy/
- SD-RAN and ROC are still pending
- Connect Pixel7a phone with srsRAN blueprint (USRP X310, band 78)
ARA
- SD-RAN
  - Checking O-RAN Alliance specifications on the E2 Service Model and SD-RAN code.
  - Exploring the potential development of xApps for spectrum sharing and network slicing
- SD-Core
  - UE barring issue troubleshooting with Ajay on Slack. Need to provide more logs.
  - SD-Core + Ericsson RAN blueprint refinement
...

1-7-25

Infrastructure
- Update Jenkins
  - Eric now has ssh access
  - Now positioned to update (base version + plugins)
  - Maintenance window mid-day (most jobs run late afternoon PT)
- Co-locate Registry & Jenkins VMs on same subnet
  - Need to sync Aether Registry with already published images on Docker Hub
  - Need to publish released images to Aether Registry (also Docker Hub as a backup)
    - This is the next step, for: SD-Core, SD-RAN, and ROC
    - No need to publish per-PR images (see item below)
    - Wait to do this until we've consolidated GitHub "publish" Action
  - Need to update OnRamp playbooks to use the Aether Registry
  - Need to ensure the Registry stays up (at one time certificate failures caused issues)
- Consolidated GitHub Actions
  - Still in-progress
  - Registry staying current for future images gated on this happening
SD-Core
- Need to review/update maintainer's group in the omec-project organization (Gabriel)
  - Add Ghislain to maintainer's group and should we still require a single approver, or should we increase to 2 approvers?
  - Need to link "Contributor" doc to Wiki (establishes the practice).
- Do we need to keep creating "per-PR" images? There are not used anywhere (Gabriel)
  - See above
- 3GPP Release Update → Good Progress!
- Working on e2e tests for changes to any NF.
SD-RAN
- Draft Release Notes
- Working on a blog and a Webinar
OnRamp
- Process to add additional blueprints (repositories/sub-modules)? (Gabriel)
  - We need a couple repositories to be created for 2 blueprints we are working on → Done
  - See https://docs.aetherproject.org/master/onramp/blueprints.html#other-blueprints for "Requirements"
ARA Deployments
- SD-RAN
  - Checking O-RAN Alliance specifications on the E2 Service Model and SD-RAN code.
  - Exploring the potential development of xApps for spectrum sharing and network slicing
- Publish SD-RAN Design papers
  - There is a preliminary "review" of the SD-RAN RIC
- There is now ARA Wiki Page (on the Aether Wiki)
  - ARA Wireless Living Lab
- SD-Core
  - SD-Core + Ericsson RAN blueprint draft ready
  - UE barring issue still pending (interacting with Ajay on Slack)

12-17-24

Remaining meetings: December 24 and December 31 - keep or cancel?
- To cancel. Next meeting Jan 7th.
ROC
- GitHub Actions to publish ROC artifacts. Status?
  - Have merged latest changes .github → to publish helm charts
  - Next step is to test run in an example repo
  - Avoided being "overly generic" → targeting ROC in onosproject repos, but can be adapted
  - Publishing images will be next
  - Want to port to the omec-project repos once done (related to other Jira ticket)
- Why it is not possible to configure OP (or OPc) and Key through GUI? (Gabriel)
  - The assumption was that the subscribers block (https://github.com/opennetworkinglab/aether-5gc/blob/master/roles/core/templates/sdcore-5g-values.yaml#L128) is a "root" provisioning that falls outside the scope of what the ROC was designed to manipulate.
  - Is the "root" mechanism sufficient/scalable? → Not necessarily (probably not).
  - Could ROC be extended to support changing setting this info? → Yes.
Infrastructure
- Jenkins performance/stability. Any update?
- Blocked on AWS credentials
OnRamp
- Version control: It would be good to tag versions/releases such that it is easier to provide support to end users (Gabriel)
  - Should be done.
  - Challenges
    - Need well-defined release process => Need dependable testing of blueprints.
      - Releases paced by "weakest" blueprint.
      - E.g., Latest SD-Core release gated by having all the blueprints work correctly.
    - Today, OnRamp releases ~ SD-Core Releases
- Subnet variables working and ready for PR (see subnets branch of various repos)
  - See https://github.com/opennetworkinglab/aether-onramp/blob/subnets/vars/main.yml for basic idea
- srsRAN Jenkins job being debugged
- Process to add other blueprints (e.g., Confidential compute, N3IWF, etc.)? (Gabriel)
SD-RAN (Woojoong)
- Release blockers
  - SD-Core chart 1.4 has an issue in RiaB (4G)
    - It pointed aether helm chart, which seems to be removed → updated it with sdcore-helm-charts
    - It was used very old SD-Core chart → bumped up the version to 1.4.0
    - UPF was facing OOMKilled → fixed by doubling memory limit
    - Too many open files issue in MME and PCRF → set fs.file-max
    - PR: https://github.com/onosproject/sdran-in-a-box/pull/207
  - 4G hardware document (4G)
    - Out-of-dated → updating now
- What is the plan to merge currently open PRs from Dependabot? (Gabriel)
  - High priority for next release, which hope to start working on after the holidays
- Why do we maintain/keep RiaB instead of OnRamp? (Gabriel)
  - RiaB currently includes 4G elements that are not supported by OnRamp
SD-Core
- Update GitHub Actions to properly handle tag creation when PRs are towards release branches (e.g., rel-1.5)
- Continue effort to move code to 3GPP Release 18
  - Created models and APIs for some specs including Nnrf_NFDiscovery that is used in the nrf_cache
ARA
- SD-Core
  - Currently checking logs to understand the UE context release issue (details posted on on-ramp slack channel)
  - Almost done debugging
- SD-RAN
  - No updates (Tianyi on travel)
- Blueprints
  - In progress → One for Ericsson radio and one for OAI radio
  - Could include ARA-specifics (e.g., for UEs, including experiments)
  - Intent is end-to-end deployment blueprints
    - related to OnRamp releases since more blueprints => higher hurdle
    - Need "owner" for every blueprint; some may be "core" and others "auxiliary"

12-10-24

TST Election Results
- Welcome Bilal, Ghislain, and Tianyi (welcome back Gabriel)
- Much thanks to Andy, Scott, and Shahbaz
Infrastructure
- Provenance of registry.aetherproject.org
  - Gabriel has admin for sdcore
    - Incremental images pushed to Aether Registry
    - Officially released images pushed to Docker Hub
  - Overall admin unknown
  - Have now identified AWS account
  - There are other projects that are no longer related
  - See Jenkins issue below
  - In the past this registry wasn't 100% reliable
- GitHub Action consolidation (reusable workflows) for omec-project
  - work is underway
- GitHub Actions for publishing ROC artifacts
  - To get input from Gabriel and Sean
  - Repos: roc-helm-charts, onos-config, onos-topo, aether-models, onos-cli, onos-api, config-models, onos-helm-charts, build-tools, aether-roc-[api,gui], onos-lib-go, onos-test, sdcore-adapter
- Jenkins performance is poor → image download seems to be an issue
  - Also publish released images to Aether registry
  - Co-locate (same subnet) with Jenkins (works this way with OpenCORD)
  - Alternative is to have paid account with Docker Hub
  - Ticket filed → https://jira.linuxfoundation.org/plugins/servlet/desk/portal/2/IT-27583
OnRamp
- Cleaning up subnets
  - Variables for core_subnet (192.168.250.0/24) & access_subnet (192.168.252.0/24}
  - Works for gNBsim
  - Testing for OAI, UERANSIM and srsRAN in progress
- Update namespace for SD-Core to "aether-5gc"
SD-Core
- Reviewed and merged ~40 PRs opened by Dependabot
- Improvements to code by properly handling possible Panics (AMF, NRF)
- Continue working on 3GPP release upgrade (Release 18)
SD-RAN (update posted to Slack)
- SD-RAN Engineering team showcased SD-RAN and Aether-OnRamp in Korean conference - went well!
- SD-RAN release
  - We bumped up the sd-core chart but it's not working with 4G OAI with SD-RAN in a Box.
  - We are investigating it - this is the release blocker, which I am mostly trying to fix..
ARA Deployments
- SD-RAN
  - Studying the RIC/SM/xApps to see how them can contribute to rural application
- SD-Core
  - Started rolling out SIM cards for field-deployed UEs to work with SD-Core
  - UE context releases after sometime and unable to register afterwards.
    - Happens after a few hours, even with active traffic; can't reattached
    - With Ericsson gNB (not observed in the past when used their core)
    - Collecting more logs to help debug

12-03-24

TST Election
- Eligible voter list (mostly) complete
- Nominations due by Friday
Infrastructure
- Any status update about consolidating GitHub Actions (omec-project)? → Gabriel to file a ticket → Done (ticket: https://jira.linuxfoundation.org/plugins/servlet/desk/portal/2/IT-27547)
- GitHub Actions for ROC → Larry to file a ticket → Done (ticket: https://jira.linuxfoundation.org/plugins/servlet/desk/portal/2/IT-27552)
SD-Core
- Merged PRs (UPF and SMF) related to TEID allocation
- Currently working on enabling cert-manager as a sub-chart for the SD-Core Helm Charts
- Continue working on 3GPP release upgrade (Release 18)
- Scalability PRs still in progress
SD-RAN
- Penn State reporting security risks in SD-RAN (also SD-Core); see report
OnRamp
- OnRamp is now using the latest version of SD-Core Helm Charts (v2.0.0)
- srsRAN Blueprint (simulation mode only) now available (integration tests & documentation still pending)
- Any thoughts on renaming the default branch to use current GitHub name (main) instead of master for OnRamp and its dependencies/sub-modules?
- Are the "dependencies" such as versions of K8s, Helm, etc. frequently updated?
- Why not accordingly update the name for SD-Core namespace instead of using legacy name (omec)?
- Integration tests (Jenkins) still plagued by timing issues (e.g., only 4 of 5 UE succeed).
ARA
- SD-Core w/ Ericsson RAN
  - Tested with 4 E/// BSes
  - Fixed IP assignment to specific UEs possible?

11-26-24

No meeting

11-19-24

TST Election
- Soliciting nominations
- Determining who qualifies to vote
Infrastructure
- Issue with srsRAN Docker repo
SD-Core
- Process for accepting PRs when optional components get "affected" (stop working)
- Chunk-tizing PRs from Purdue for scaling
- Propose and vet designs before submitting PRs (
  - especially when need to take e2e issues into consideration
SD-RAN
- Aether On-Ramp has 1.5 SD-RAN helm chart now
- Fixed user doc site fully! - multiversion had an issue; it only showed master but now it showed old version too.
- Documentation work on going
  - Release note draft ready - will ask a review sometime this week
  - HW installation - engineering team - Hyunmin - is leading to write it up
  - Aether On-Ramp for SD-RAN - draft is ready
- Engineering team is debugging E2 decoding issue when testing with Viavi E2 simulator
- Starting to turn focus to next release (see wiki)
OnRamp
- Process for testing SD Core Helm Charts v2.0.0
- Completed srsRAN blueprint (simulation mode only)
  - We (Aether) are creating the required Docker images
  - Address some intermittent issues (e.g., AMF crashes)
  - Add support for gNB w/ USRP
- Cleaning up subnet configuration & usage (in progress)
  - Document the Linux bridges being used
- Ansible: host.ini (sudo password)
ARA Deployments
- SD-RAN:
  - Working in outdoor.
  - Checking how the RIC supports URLLC, slicing and spectrum sharing
- SD-Core:
  - Throughput stable with E/// RAN
    - Tweaked Core-to-gNB transport (best guess at the moment)
  - SD-Core pushing around 850Mbps to E/// RAN and UE getting 600Mbps on avg.
    - May have been environmental factors; will retest
  - Enable DPDK and test throughput again.
  - Will also test with multiple gNBs

11-12-24

SD-Core
- Updated Helm Charts with several improvements (use yaml extension, configurable path for config files, among other)
- SMF/UPF: TEID allocation
- IOS-MCN
  - Proposal for multiple DNNs
  - SD-Core reset: AMP
Aether OnRamp
- srsRAN blueprint in process of merging
ARA
- SD-RAN
  - Working on outdoor testing of SD-RAN with OAI + USRP SDR.
- SD-Core w/ Ericsson RAN
  - ARA team, IL and Ericsson will have a debugging session tomorrow

11-5-24

TST Elections
- Discuss & Vote on TST Governance Document (unanimously accepted)
- Encourage nominations
Infrastructure
- Sphinx documentation (hosted on static.opennetworking.org basically)
  - Migrate to ReadTheDocs? (SaaS for Sphinx documentation)
  - link: https://about.readthedocs.com/
  - Standard throughout LF
  - Community (free with ads), Gold (no ads, $5.00/month)
  - Alternative is to publish on GitHub (llp)
- DNS
  - In process of moving and migrating off of referencing *.opennetworking.org sites
    - Trying to avoid downtime or docs sites being unavailable during transition.
- LF Insight needs to be updated to report stats for all three GitHub Orgs
Aether OnRamp
- Jenkins jobs now successfully running for all Blueprints
- Rich debugging opportunity – looking for volunteers
IOS/MCN Presentation
- Overview Deck
SD-Core
- Enabled secure communication (TLS/https) for SBI by default in Helm Charts
  - Pending: i) Secure GRPC communication; ii) Use a CA instead of self-sign certificates
- Alignment for configuration across all NFs (Docker image, configuration arguments/flag, parametrize path for config file, etc.)
ARA
- SD-RAN
  - Working in sandbox with OAI and USRP SDR; moving to outdoor testing.
- SD-Core w/ Ericsson RAN
  - Working with Gabriel and Suresh to get a common time to have a meeting with Ericsson team to troubleshoot throughput instability issue observed.
...

10-29-24

Infrastructure
- DNS names cleaned up
  - docs.sd-ran.aetherproject.org, docs.sd-ran.org, and docs.sd-core.aetherproject.org work as expected
    We don't have control over sd-core.org
    sd-ran.org and sd-ran.aetherproject.org redirect to aetherproject.org
- Rate-Limits
  - For information only – OMECPROJECT Docker Pull Counts
  - No progress on plan
- Certificates
  - Need authorization for Slack channel update to #aether-tst – pending
- DNS Sites
  - Will be moving *.opennetworking.org to *.aetherproject or *.volthaproject domains as pertinent
  - May require additional community and development education.

Does anybody use:

wiki.opencord.org

static.opencord.org

charts.opencord.org (needs attention --llp)

guide.opencord.org

opencord.org

static.opencord.org

wiki-archive.opencord.org

www.opencord.org

- Slack permissions → Needs cleanup, but only workspace owner can demote admins

OnRamp
- Blueprint Quick Reference
- All Jenkins jobs are working, except Multi-UPF
- Loading Models into ROC remains flaky (FYI)
- When ROC controls the core, after a Core reset (restarting all the pods), e2e tests fail (performance metrics also reset)
SD-RAN
- SD-RAN Doc repo now has github action to publish the doc!
- 1.5 Doc is WIP especially for hardware installation and release not (hope it will be done this week)
- Will announce internally, then do a public announcement (blog, webinar, etc)
- Working on onos-e2t integration with Viavi simulator actively. We got the error message and now try to fix it.
- Start planning 1.6 release
  - Bump up dependencies' versions and go version
  - Re-design APIs
  - CCC service model (PR ready, but not fully tested yet)
  - Viavi simulator integration
  - RAN sim v2.0
  - OAI 5G integration (use latest OAI code)
  - non-RT RIC integration (1.6 - just for PoC)
  - User documentation
SD-Core
- Enable ZUC algorithm (NEA3, NIA3) at NAS level
- Enable HTTPS/TLS for SBI interface in sdcore-helm-charts
  - Parametrize path for certificates in NFs
- NFs go into an infinity loop when the webui pod is restarted
  - Canonical provided a solution, which was merged in most of the NR (PR for UDR still pending)
- Clean up deprecated linters from golangci
- Enable authentication and authorization for WEBUI
- Breaking Bilal's big PR into chunks and working through them
ARA
- SD-RAN
  - Work on SD-RAN with OAI (2022.w51). Connected the RIC to OAI gNB and SDR but solving issue to connect to OAI nrUE.
- Ericsson GNB Performance Investigation: → We tested various use cases with different core and find below our observation (all tests are UDP tests).
  - iperf with UERANSIM against aether SD-Core:
    UL throughput: 986 Mbps
    DL throughput: 986 Mbps
  - iperf with Ericsson GNB against aether SD-Core:
    UL throughput: 111 Mbps
    DL throughput: 144 Mbps
  - iperf with Ericsson GNB against open5GS:
    UL throughput: 109 Mbps
    DL throughput: 143 Mbps
  - We tested using fast.com (which uses TCP) and we were getting around 144Mbps.
  - Initially get expected xput with SD-Core, but a packet drop impacts the xput rate (that doesn't happen w/ Ericsson core)
TST Elections → Coming soon.

10-22-24 (Today's Agenda)

Infrastructure Update
- Expired certificates updated → How do we keep this from happening again?
  - Warning to be sent to aether-tst channel
  - By using local Helm Charts, OnRamp can avoid this issue
- DockerHub rate limit → Now impacting daily integration tests
  - Publish all images to Aether Registry (official releases still go to DockerHub)
  - But... have to keep the Aether Registry available
  - To put together a specific plan of action (daf, llp, gabriel)
- DNS
  - https://docs.sd-ran.org/ works
  - https://docs.sd-core.org/ does not work (neither does sd-core.opennetworking.org)
  - Also want sd-core.aetherproject.org and sd-ran.aetherproject.org
  - To put together a specific plan of action (llp, daf)
OnRamp Update
- Refactoring PR to be merged later today → Watch for updates to documentation
- UERANSIM blueprint includes data plane test
- Why do we need Helm Charts from opencord when deploying the 5GC?
  - I don't think we do; plan to remove.
- Re-org docs to highlight blueprints
SD-RAN Update
- Adding Github action to SD-RAN doc repo
- Updating SD-RAN doc - especially hardware installation part as it is out-of date
- Latest Helm Chart (1.4.x) is actually for v1.5
  - going to re-number as 1.5.0 Helm Chart version to match release
- O-RAN plugfest - Viavi integration; kick-off meeting this week
SD-Core Update
- Renamed default branch for all repos to "main"
  - Some repos already used "main" as default
  - For the other repos, everything got automatically updated (e.g., PRs)
- UPF-Adapter code update in progress (enable metrics)
  - Helm Charts are already updated
- Review all outstanding PRs
- gNBsim paper under review
ROC Update
- GitHub Actions?
- Still pending
ARA Update
- SD-RAN deployment
  - Begin setting up SD-RAN, progressing toward running OAI 2022.w51 with SD-RAN.
- SD-CORE deployment
  - Trying to improve throughput from Ericsson gNB
    - Expecting 700-820Mbps
  - Suresh at Intel Labs is focused on performance
    - He's seeing no issues (getting around 1Gbps)
    - Connecting Suresh with Joshua on Slack
    - Easiest to troubleshoot using UERANSIM
- Exploring SMART-5G & Rakuten POC effort (09-24-24 meeting notes)
- Miscellaneous items
  - Interest in "ARA Blueprint" advised by Larry
  - Plugfest at ARA OTIC (O-RAN Testing Centers)
  - June 2025 gathering of wireless and AgTech industry leaders (via NTIA ACCoRD project & likely AraFest'25)

10-15-24

Infrastructure
- Docker Hub organizations (residual from ONF days)
- Limit of three seats per org
  - Need one for "lfreleng"
  - Need "onfauto → currently pushes SD-Core (and other) images
  - Open Third Seat → assign to a TST member (currently Gabriel)
- Three "quiet" orgs → xosproject & onfinfra (remove), opencord (check with Voltha)
- Five orgs created but never used (keep "aetherproject" & "openaether" remove the other three)
SD-Core Update
- Release new Helm Charts using the latest versions of the images, including the upfadapter image
  - Code for the upfadapter was removed from the smf repo
- Reviewed and merged most of the open PRs (Currently, 14 PRs open: 5 are drafts, 5 dependabot PRs have issues due to protobuf update/version)
- Daf will help us renaming the default branch to "main" and update the GHAs to point to "main"
- omec-pod-init and omec-cni repos still pending to create
OnRamp Update
- Pipelines for integration tests now in GitHub
  - Groovy Pipelines: https://github.com/opennetworkinglab/aether-jenkins
  - Jenkins Dashboard: https://jenkins.aetherproject.org/
  - Watch failures on Slack: #aether-jenkins channel
  - Still a work-in-progress
- Stability remains an issue
  - DockerHub rate limits
    - Anonymous users: 100 pulls per 6 hours per IP address
    - Authenticated users: 200 pulls per 6 hours
    - Docker Pro and Team subscribers: Up to 5,000 pulls per 24 hours
  - VM resources?
    - Base (4vcpu+16GB). Trying (8vcpu+32GB)
  - TIme to download images from DockerHub?
    - Could shift to Aether Registry (on a VM inside our AWS account)
    - Docs, Registry, Jenkins, Docker Builder/Executor (may not need since we've moved off Jenkins CI tools)
- ueransim blueprint is now using the latest version of SD-Core Helm Charts
  - Issues starting the gNB and UE as Linux processes? → Was working at one time, but...
  - The Ansible playbook is flakey(did work for 1 of 3 instances – UE/gNB pair of processes)
  - ARA also seeing problems
- srsRAN blueprint status?
SD-RAN Update
- ...
ARA Update
- Waiting on v1.5 release of SD-RAN
- Still seeing throughput issue for Ericsson gNB and SD-Core
...

10-8-24

SD-Core Update
- Immediate Goal → Merge Scalability PR
- Replacing "logrus" with "zap" logger was completed
- Created new images
  - upfadapter was moved from SMF to its own repo
  - upfadapter image not getting published (https://jira.linuxfoundation.org/plugins/servlet/desk/portal/2/IT-27307)
- Settle on main rather than master → so consistent for automation
- There are currently ~45 PRs open (30+ dependabot), rest from Canonical
SD-RAN Update
- Immediate Goal → Complete v1.5 release
  - Documentation CI was not configured - figuring out how to do it (checking Aether doc repo)
    - Both SD-Core and Aether have moved to GitHub actions
    - Main change to docs is the release notes
  - No other engineering task other than documentation
- Try to participate O-RAN plugfest fall 2024 (hosted by LG U+ - Korean TelCo company)
  - Topic: Viavi RIC tester & SD-RAN E2T integration
- Participated ICT Challenge 2024 (Korean conference)
  - Showcased SD-RAN (potential 1.5 release code) E2E test with 4G SD-Core, OAI 4G, USRP working
- Submitted one conference paper (Korean domestic conference KICS 2024 Fall) - SDRAN E2E test topic
- Wish List: Combined SD-RAN + OAI 5G blueprint (part of v1.6)
  - Open Question: Latest OAI code vs Older version with our E2 Agent
OnRamp Update
- Immediate Goal → Reliable daily builds for all blueprints
- Auditing (and cleaning up) integration tests
- Improvements to UERANSIM blueprint
  - Still need to debug issue with latest charts
- Other "robustness" improvements in progress
- Wish List: an srsRAN blueprint (modeled after the OAI blueprint)
ROC Update
- Immediate Goal → Reproduce CI tooling
- Plan is to jump straight to GitHub Actions (Publish is still missing)
ARA Update
- Immediate Goal → Deploy Aether in various scenarios
- Waiting on SD-RAN v1.5, then will integrate with OAI 5G
- Working on Ericsson gNB ↔ SD-Core
- Participating in ACCoRD (NTIA funded) as "neutral" party (Radisys gNB + SD-Core)
...

10-1-24

SD-RAN Status
- SD-RAN 1.5 code is all ready
  - OnRamp has been upgraded to deploy
- SD-RAN doc could not be published. Checking the CI.
- Now Planning 1.6 release
Infra Status
- Jenkins
  - Original "Static VM" jobs have all been disabled
  - Full set of "Dynamic VM" jobs are being debugged
- ...
SD-Core Status
- Scalability PRs are being evaluated
  - GitHub Actions flagging some issues
- Start replacing Logrus with zap logger
- Fixed issue with gnbsim profile4 (parallel execution)
- Several other patches accepted this last week
  - From Canonical
  - From IOS
- Release Plans
  - Next: Scale + Logger
  - Future: Rel 18 upgrades
OnRamp Status
- SD-RAN Blueprint updated for v1.5
- UERANSIM Blueprint has been refactored
  - Need to update the documentation
- Additional refactoring still in-progress
Radio Interoperability
- Ericsson (AIR 6488) working, but suboptimal throughput
...

9-24-24

SD-Core
- Pending "Scalable Core" PR (Bilal)
  - NRF caching (remove bottleneck)
  - Enhanced load balancing (UE-based caching)
  - Also includes several latency/bug fixes
  - Touches nearly all Network Functions
  - Plan is to review with Gabriel and Ajay
  - Changes on GitLab; will issue PR to GitHub
    - Gabriel will postpone log-related changes until after merge
- Progress connecting Ericsson gNB to SD-Core (Gabriel, Joshua)
  - Fixed issue with NGAP field
  - Now have N2 connection → Have successfully established PDU session
  - Now tripping over User Plane issues
- Working on Release upgrade (Gabriel)
- There has been a steady stream of PRs from IOS team
LF News
- free5gc project has joined LF
- Talk to Timon about "harmonization" opportunities
- Coincidently, today we finished "repo independence" from original fork
- API for Release upgrades could be a useful point of cooperation, but free5gc uses their private code generation tools
OnRamp
- Issues with ueransim blueprint (Gabriel)
  - There might be "secret sauce" in Jenkins job
- Kubernetes config PR committed (see Allocating CPU Cores)
- Modest refactoring of Ansible variables still in progress
- OAI Blueprint → Improperly set config flag (Joshua)
Infra Update
- Reminder to use Jira to report bugs and request features
- 2-server Jenkins jobs still not running correctly → Might be running (changing output to determine)
- Working on stability of other jobs
SD-RAN Update
- New Helm Charts released: 1.4.143
- Have not heard any news on 1.5 release notes
O-RAN SMO / non-RT RIC integration? (in addition to Aether Management Plane?)
- Status & roadmap for SD-RAN A1 interface? O1 interface?
  - Expected people effort needed (if any)?
- Candidate SMO / non-RT RIC that is more compatible with SD-RAN? OSC SMO?
- Aether Strategy has been to use ROC (AMP) as "RAN SMO"
  - Prototyping work was done, but not carried forward
    - PRs exist, but not merged
    - Depend on CCC Service model, which is yet-to-be merged.
  - The A1 interface for SD-RAN does exist
- There is also SMART-5G work that as been leveraging open source SMO, rApps and O1/A1 interfaces:
  - https://aetherproject.org/smart-5g/
  - and a new PoC starting that is building a PoC for Rakuten leveraging open source (rApp and O1)
    - https://drive.google.com/file/d/1gLbHsGafIdpwfMsaWOr113Ijf7pKPT-H/view
Miscl.
- free5gc & commercial version

9-17-24 (No Meeting)

9-10-24

SD-Core Development Priorities
- Reminder: Current Roadmap
- Upgrade to Release 18
- ARA Focus
  - Get SD-Core working with OAI (done, being demo'ed at OAI Summit)
  - Also want to demo Core with other radios
    - e.g., Working with Ericsson radios with Aether Core
  - Scalable Control Plane
  - Research more focused on SD-RAN topics
    - Scheduling & Resource allocation issues
    - Low latency & power control)
    - Working on xApps
- Canonical Priorities
  - Keep up with 3GPP
  - Slicing support
  - Working with other radios (e.g., OAI)
  - Canonical is contributing
    - additional integration mechanisms
    - reliability/hardening (e.g., restarting)
    - removing backdoor dependencies of NF on database
- ...
SD-RAN v1.5 Release Status
- Latest SDRAN charts test done - all working!
- Publishing the latest Helm chart now.
- Finalizing the documentation
Infra Update
- Jira access
  - ALL product work, new features, or issues should be reported and tracked on the Project Aether site:
    - https://jira.aetherproject.org
    - Please use appropriate component for where work is needed.
    - Infrastructure or Release engineering associated with the Jira.aetherproject.org environment itself:
      - https://jira.aetherproject.org
        Component: Jira Setup
  - Other generic Infrastructure issues:
    - https://jira.linuxfoundation.org
    - When creating a ticket,
      - use component 'Release Engineering',
      - Product will be Aether
  - Note – While I am the primary infrastructure engineer for Aether, other members of the LF team may fill in or back up at times. This is why generic infrastructure issues should be filed in the LF Jira, rather than the Aether-specific one.
- Jenkins reduction
  - Jobs significantly scaled down ...
  - All Aether OnRamp jobs EXCEPT for 2Servers-22.04-default-charts dynamic
  - Expect to have 2Servers-22.04-default-charts this evening dynamic
  - What are necessary ROC CI jobs
- Jira/Confluence
  - Problems with initial access have been seen, but seem to go away after 2 or three attempts
  - Any issues with access?
- Open Tickets::
  - IT-27166 - Jira Dashboard
  - IT-27178 - Jenkins Clena Up: OnRamp Tests
  - IT-27179 - Jenkins Clean Up: ROC API Tests
  - IT-27105 - Bring omec-cni and omec-pod-init from Gerrit to Github (OMEC)
  - IT-27168 - Forwarded emails being discarded by gmail due to DMARC
  - IT-27116 - Discourse Forum for P4 timing out for new logins
OnRamp Update
- Updated k82-related PR expected soon
- Global "aether" section of vars file in-progress
- New "Blueprint Guidelines" now in Doc.

9-3-24

New User Communities
Infrastructure
- Jenkins
- Jira
  - Avoid https://lf-aether.atlassian.net/ branding?
  - Find default dashboard?
- Wiki
  - Avoid https://lf-aether.atlassian.net/ branding?
  - Set Overview page as default home?
- DNS for Docs
SD-Core
- Created upf-adaptor repo (moved out of smf); fixes metric problem
- Found issues with gNBsim stats
- Added dependency check of kafka to amf and smf
- Merged patches from IOS-MCN; other "proposals" being discussed
- Issues trying to run gNBsim against Magma (similar issues with OAI Core)
  - related to service request handling
- No progress on Rel-18 upgrade
SD-RAN
- Not much progress on v1.5 release
- Shared OAI code with ISU; should work now
- Testing in prep for release the end of this week
- Writing documentation for release (how to use OnRamp for SD-RAN)
- Once 1.5 is ready, will start integrating CCC E2 Service Model (merged into master)
OnRamp
- PR related to k8s configuration still pending
- Evaluating "global" section & design guidelines

8-27-24

SD-RAN
- Woojoong will provide an update later
  - Will update on Sept W1 deadline
- Iowa state univ
  - could not build the E2 Agent from OAI from 2022w51 tag
  - Kushal pushed a patch - could build E2 RIC agent and 5G OAI with it
  - Should help for 1.5

SD-Core
- Created new images (with Canonical)
- Published new helm charts
- Gabriel has tested with OnRamp
  - Would like Larry to verify them on OnRamp
  - Test them on Jenkins jobs
- One of the fixes in GNB Sim allowed Bilal to move forward
  - Found the next issue - still investigating
  - Had not tested with Magma
  - Also testing GNB Sim across multiple 5G cores
    - e.g Free 5GC, Open5G
- Fixed a metrics bug in SD-Core by downgrading Kafka
  - Will look to upgrade Kafka and Mongo to latest next
- Replacing Logrus with Zap logger
  - Canonical may be able to help with this
- Discussion about e2e test on PRs
  - No progress on this
- Waiting on repo omec-cni, omec-pod-init (to rename)
  - Gabriel waiting on Daf to create these GitHub repos

OnRamp
- PR related to k8s configuration pending - meeting with Gabriel, Suresh and Larry to sync up on this

Infrastructure
- Confluence
  - The wiki.aetherproject.org site is now read-only as we migrate the contents to the Atlassian Cloud
  - The old site will remain available in read-only mode at wiki.archive.aetherproject.org for at least one month.

ROC
- No progress on updating Wiki - Sean to continue on this

8-20-24

SD-RAN
- Working on v1.5 release
  - Working on documentation and release notes
  - To include OAI 5G RAN
    - includes "ONF" E2 agent (not OAI's)
    - implies need to rebuild OAI images (since E2 agent is statically included)
  - Not include update to golang
  - Expect announce 1st week of September
  - E2SM → Will add O-RAN RC Service Model
    - Backward compatible with the old ONF Service Model
Infrastructure
- Jira
  - Production site available at https://jira.aetherproject.org
    - Still working out Anonymous creation of issues
  - Permission Question:
    - Currently four levels of access
      1. Anonymous (users without Atlassian accounts or not logged into them) can Browse and Read existing Issues but not modify anything
      2. Any Logged in user can basically change almost any field in an issue, and invite people to join the site and become a user
        When invited, specific domains are automatically approved when invitee makes the request
        Basically, any TSC user's domain (that is NOT public (ie, not gmail, yahoo, aol, ...))] is currently automatically approved.
      3. aether-jira-admin users can update group and user permissions
      4. admin can do anything.
    - QUESTION: Any need now to restrict (beyond being logged in as an approved Aether Jira user) things like:
      - Transitioning a ticket between states (including closing)
      - Mucking with assignee
      - editting fields beyond comments
      - → Taking a wait-and-see approach
  - Known TODOs
    - Figure out how to let Anonymous users create new tickets
      - Currently looks like a separate form to embed on an appropriate webpage that will then go create a ticket based on provided input
    - TST Guidance: Lowering the barrier to creating a Jira account >> creating Anonymous Tickets does not need to be supported
- Confluence
  - Test migration appeared to complete successfully
  - QUESTION: Can I perform the formal migration from existing Confluence DataServer to Confluence Cloud between 12:00 noon Wednesday(8/21) and 12:00 noon Thursday (8/22) (PDT)?
    - Will require read-only Confluence installation at start of process until completion.
    - Upon completion, existing wiki.aetherproject.org will point to new site
      - Previous site will be maintained for approximately 30 days as an archive/reference site and then hopefully deleted.
    - I would hope that people can spend a 30-60 minutes reviewing the site after completion and ensuring things look okay. Hoping to have acceptance of new site by next week's TST.
    - Post note on migration to #aether-dev channel
- Jenkins
  - No progress as yet
  - No roadblocks
- SD-RAN DNS
  - No progress from last week
OnRamp
- PR related to k8s configuration pending
- Still debugging monitoring system (metricfunc failing)
Question to Group: Anyone connected to "Aether Using Nephio" project?
SD-Core
- Incorporating the static checks for linting with other PRs
- No progress on logging
- Discussion about e2e test on PRs
- Waiting on repo omec-cni, omec-pod-init (to rename)

8-13-24

SD-Core
- Check with Ajay again about an SD Core deployment mode (below)
- Status update:
  - Removed in gnbsim dependency from the CommonConsumerTestData. There were a few functions calling the CommonConsumerTestData but these functions were not used. So, CommonConsumerDataTest repo is now archived
- golangci-lint not detecting all issues. For example, staticcheck is enable in golangci and it successfully completes. However, if we run staticcheck by itself, it finds several issues that are not found through golangci
- We use logrus (https://github.com/sirupsen/logrus) for logging, where the last commit is from June 2023. This might mean that the project is inactive (none is actively maintaining it). Based on this, I explore a few options and zap (https://github.com/uber-go/zap) seems to be a good alternative. As per its description zap is "Blazing fast, structured, leveled logging in Go". My suggestion would be to replace the logger. What are your thoughts? If agree, who can volunteer to work on this?
- It would be good to check whether golanci-lint catches issues when using zap
- GitHub Action (or Jenkins job) to run E2E tests for PR verification/validation
  - This could mean the use of a simplified version of AiaB
- Bring omec-cni and omec-pod-init from Gerrit to GitHub (omec-project organization)
- A "Scalability" PR is in progress
- Metrics currently not being displayed by Grafana → dashboard spec has fallen out-of-date
OnRamp
- OAI Blueprint has been published: https://docs.aetherproject.org/master/onramp/blueprints.html#oai-5g-ran
  - Plan to demo at ARAFest'24 (https://arawireless.org/arafest24/)
- PR to add "resource variables" to Kubernetes configuration is under review
Infrastructure
- Jira
  - Setting up production site – anticipate availability EOD Wed.
    - Site will be lf-aether.atlassian.net → jira.aetherproject.org
  - Accounts will be initially all invited members of the TSC
    - Please file a ticket (in lf-aether-test.atlassian.net) if you want to pre-seed other accounts
- Confluence
  - Migration to Cloud (from existing LF-hosted site)
    - Test migration of existing content (from current wiki.aetherproject.org) this week
      - No downtime
      - Will request (on #aether-dev?) BETTER SUGGESTION? (Yes – largest subscription) for folks to review test site for validity
    - Ideally, request production migration next week's TSC
    - Upon approval, site will be offline approximately 4 hours
    - No functional changes anticipated.

- Jenkins
  - All QuickStart jobs are now dynamic:
    - 20.04 (2 succeed, 4 fail)
    - 22.04 (All 4 succeed)
    - Report failures to #aether-jenkins slack channel
  - Will build 24.04 versions of all 22.04 jobs this week
  - Assuming 24.04 job success if priority
  - 2Server jobs are still static, anticipate converting to dynamic as much as possible by end of month:
  - Are all jobs still desired?:
- DNS for Docs
  - docs.sd-ran.org as desired
  - docs.sd-ran.aetherproject.org
    - not secure, needs cert
  - docs.sdran.aetherproject.org
    - Not resolving, invalid config on server
  - Issues above on tomorrow's schedule, anticipate success EOD Thursday
SD-RAN
- ...
ROC
- ...

8-6-24

SD-Core
- Should the sdcore-helm-charts work "out-of-the-box"? That is, should the users be able to deploy the SD-Core by running "helm install" command without having to use/rely on Aether-OnRamp (or AiaB)? [GA]
  - There are somethings need to be modified first e.g. UPF is deployed in DPDK mode default (most performant)
  - It depends on your hardware - there are a number of pre-requisites for this.
  - AiaB and OnRamp deploys in AF_Packet mode (in additional values file)
  - Q for Ajay & SD-Core team - should the chart use AF_Packet by default - we should strive for a positive experience when bringing it up regardless of method
- Moving towards SD-Core 3GPP Release 18 (Gabriel)
  - Implementing on SD-Core Roadmap
  - Executing on parts 1.a and 1.b among others
SD-RAN
- Release 1.5 - in Aug with no dependency update - it is too much tightly coupled with other ONOS components especially for onos-api. (WK)
  - SD-RAN components are on Go 1.17 to 1.19
  - Option 1
    - Continue trying to update everything. This is becoming increasing difficult because of common dependencies
  - Option 2
    - Can we separate SD-RAN ric from uONOS under Aether or separate Github workspace? - in that case, we could say SD-RAN inherits a design/code from uONOS.
      - While doing that, fix e2sm and add new ransim.
      - Prevents breaking other components
  - Woojong will talk with Larry after SD-RAN 1.5 release to try to make a decision
- Talked with Awias today (Prof in Univ of Glasglow): great discussion and will keep doing it to find out the new research topic.
  - ARs: share the SD-RAN tutorials to him.
  - Possible research topics
    - Secure comm
    - AI/ML based self healing of RAN
LF collaboration updates (Timon)
- RIC evaluation project kicked off
  - Funded, Rimedo is contractor. Has kicked off. Final report in December 24
- Discussing making Aether onramp 'blueprints' nest inside the notion of LF 5g-superblueprints to provide core and various RAN options
  - LF wants to consolidate around SD-Core - the core of choice for LF (replacing Magma from Meta which is waning)
  - 5g-superblueprints are very broad in scope
    - Many different topics e.g. Edge, Cloud
  - We should try to package a proposal on embedding SD-Core (and Aether OnRamp (as a blueprint)) in to these 5g-super-blueprints and present this to LFN TST
ROC
- Continue updating Wiki on the update of ROC model (Sean)
Infra updates
- No new reports - Daf not present

7-30-24

Infrastructure updates:
- Jira Demo
  - No issues reported in Jira setup.
  - Did anybody play around?
- Jenkins
  - 12/13 QuickStart jobs converted to Dynamic
  - Yesterday - 3 failures — Today, 12
    - gnbsim summary not being generated → likely due to VM setup not being stably finished
  - NOTE: QS-2004 and QS2204 views — *EC2 jobs are dynamic
- Docs
  - CNAMES for docs.sdcore.aetherproject.org & docs.sdran.aetherproject.org → Done
  - What's next to serve content from those names?
SD-Core Status
- Coordinate interface changes
- SMF updated to use N4 interface (moving off OMEC's PFCP)
SD-RAN Status
- Aether nRT-RIC not 100% compatible with O-RAN E2 Service Models
- Multiple flavors and customizations of E2SM
- Also impacted by ASN.1 encoder (ambiguous standard)
- E2 Agent needs to account for this / E2 Agent currently statically linked into OAI images
ROC Status
- Document the e2e process for modifying the Aether API
- GitHub Action to publish → Big issue is testing before publishing
OnRamp Status
- OAI blueprint done (documentation available later today)
  - pulling in docker images from dev branch
  - some cleanup to do (deps/aether-oai → deps/oai)
  - upper/lower-case mix on make targets
  - two license files
- SD-RAN + OAI blueprint discussion
  - optionally include ORAN service models (E2 agent)
  - need to substitute our own E2 agent (inside Intel, but not yet released)
  - to be include in v1.5 release
  - may need to build and publish our own OAI images
- Should be possible to add srsRAN in pretty much the same way
- Scalability blueprint proposal? → ask for comments this week
  - Timing of scalability updates with Rel-18 updates

7-23-24

Infrastructure Updates
- Jira Demo
  - Testing playground → lf-aether-test.atlassian.net
  - All should have invites, if not, please Slack me
  - Component: "Jira Setup" for issues/features/requests
    - Threshold for action?
  - NEEDED:
    - Project Lead (Default owner) for each component:
      - OnRamp: Larry, Bilal
      - SD-Core: Ajay, Gabriel
      - SD-RAN: Woojoong
      - ROC: ???
      - Smart5G: Alex
- Jenkins
  - ROC and Aether-Onramp jobs only
    - Quickstart 20.04 jobs are (mostly) DYN → Still working to make robust
  - All others stopped, or moved to Github Actions
- GitHub:
  - QUESTION: Should I explore building a dashboard to easily see github Action statuses?
  - Still need Publish Action for ROC
SD-Core Status
- Reviewing a large PR on PFCP
- Just 4 outstanding PRs
SD-RAN Status
- Updating Go modules
- Found gRPC not working w/ latest Go (requires design work)
OnRamp Status
- Adding support for USRP (going to test today)
- Have a plan to release scale updates; to vet with Ajay and then publish for TST
ROC Status
Iowa State bringing up Aether (ARA Testbed)
- Integrating Open Source RAN with Aether's Core
- Bring questions to TST call and post to #aether-onramp channel on Slack
Have Job Templates for JJB
- For QuickStart and 2Server jobs
- Will open it up soon
Aether being used in security research

7-16-24

Infrastructure Updates
- Jira → Progress stalled; hope to have a demo next week
- Jenkins → Reduce resource usage
  - Migrate from static machines (always up) to on-demand machines
  - Clean-up across the board
  - Disabled most of the SD-Core & SD-RAN CI jobs (now done using Actions)
  - Only remaining CI jobs are for ROC
  - Expectation that we'll have things in order by mid-August
  - daf and Kunal to work together toward this (setting up a call)
- GitHub Actions (for CI)
  - SD-Core → Done
  - SD-RAN → Done
  - ROC → Actions to Publish still todo (Sean to reach out to Gabriel)
    - Also need to grow the set of people that can modify/maintain ROC
    - Suresh has gone through the process (will send his notes to Sean, who will curate)
    - Plan is to add a doc to to the wiki
- Mailing Lists
  - TST public vs private
    - LF best practices is for official TST list to be limited to TST members
    - We still have the TST slack channel for "community" discussions
SD-Core Updates
- Canonical has issued some substantial PRs
- Currently being reviewed and merged
SD-RAN Updates
- Woojoong could not join today
- Currently, we are upgrading gRPC version for dependabot PRs
- It requires onos-api update to use latest gRPC / Protobuf compiler
- University of Glasgow looking to contribute and coming up to speed
OnRamp Updates
- Looking into adding an OAI blueprint
- Looking for opportunities to add our blueprints to SLICES 5G effort
- Plan to publish "Research" Helm Charts for scaling the Core CP
  - Different images for now
  - Will integrate changes into source over time
  - Bilal writing a planning document
JJB Updates
- Job creation with single JJB definition - Done
- Job creation with separate yaml and pipeline definition - Done
- Strategy is to do this for one job; will need to template to support multiple jobs

7-9-24

SD-Core
- PCF code merged.
- Outstanding PRs from Canonical
  - Webconsole
  - SMF - PCF stack change to go-pfcp open. (Tested compatibility with BESS UPF and eUPF (eBPF based))
    - UPF adapter is not considered in the above activity. We can perhaps do it once SMF changes are final/ready.
  - AMF - AMF without mongodb.
  - AUSF
- Dependabot : PRs were merges yesterday.
- UPF debugging
  - Youtube worked well, but iperf was going to 0 throughput. Shankar from community was using AIAB, asked him to use onRamp.
  - Recommendation is not to use GRO to get good performance. By default it's enabled. Suresh is experimenting with GRO.
- UPF: minor improvements in progress.
SD-RAN: No representative.
QA/Validation Team - No representative

7-2-24

SD-RAN update Woojoong Kim (Deactivated)
- Engineering team completed CI migration from Jenkins to Github action done
  - link
- Release update
  - In July - 1.5 release
Jenkins Update Kunal Kapoor
- New group command issue:
  - Worked with Bilal, fixed now.
  - All the native tests (onRamp deployment tests) are fixed.
- JJB
  - Upgraded JJB from 3.2 to 6.0.
  - Caching issue still persist.
SD-Core
- 3gpp Spec Upgrade
  - SBI interface upgrade.
  - go-pfcp:
    - SMF using local stack
    - Use go-pfcp in SMF.
- WIP PRs
  - Run AMF independent of mongodb.
  - Webconsole/north bound interface of SD-Core - Static Page.
Report on strategic alignment with LF (Timon)
- Core - agreement that sd-core becomes the de facto core for LF
  - added to the 5g super blueprint
  - technical - LF wants to see Nephio support
  - marketing effort - packaging/doc/presentation/website
- RAN - RIC evaluation project
  - Draft SOW:
  - assembling a review committee

6-25-24 (Topics for This Meeting)

Aether Jira Proposal: Aether Jira Proposal
- Tweaked proposal and accepted as "good enough to get started"
- Should be ready to play with next week
Github org renaming: https://docs.github.com/en/organizations/managing-organization-settings/renaming-an-organization
- More representative → Mostly focused on "opennetworkinglab" (should be "aether")
- May break external links, but now is a good time to change since we're creating new docs/pages
- No objection...
Slack vs Element (https://element.io/)
- Limiting history to 1 year (affects "grandfathered" workspaces like onf-community)
- Proposal: prune onf-community back to "aether" and accept history limit (take up with voltha and p4)
Code of Conduct ((https://lfprojects.org/policies/code-of-conduct/)
- Look through the LF standard and decide at a future meeting
Jenkins Status
- ONOS GitHub Actions
  - Actions that run for PR (ROC done)
  - Actions that run for PR (SD-RAN done)
  - Not yet publishing new Docker images (in progress)
- OnRamp Integration tests
  - JJB Related: Jobs with same name as the one created earlier, are not getting created again.
  - Native Tests: All the Jenkins jobs are not able to go beyond aether-5gc-install step. → "new group" command hanging
omec-pod-init and omec-cni repos Gabriel Arrobo
- These repos are in Gerrit → Why? Not updated in years. Move to omec GitHub? → Yes.
TST Meetings in July → Will remain on the schedule and expect them to happen (with some people missing)

6-18-24

DCO (instead of CLA) in place Larry Peterson David Ferguson (Deactivated)
- See Org README here: https://github.com/opennetworkinglab
- Will current ONF/CLA be removed from the repos/checks? Gabriel Arrobo
- Add more helpful info about DCO so new users know what do to.
  - Org README should explain and link DCO.
  - David Ferguson (Deactivated) to look into "augmenting" the error message
Configuring new Jira Larry Peterson David Ferguson (Deactivated)
- Archiving old Jira
- Proposal
  - Projects: SD-Core, SD-RAN, ROC, OnRamp, Smart5G
  - Workflow: New, Assigned, Resolved, Reopened (start simple)
  - What Fields: Description, Status
  - Hopefully will demo at next week's TST
GitHub default branch should be main instead of master. Ajay Lotan Thakur
- Master is old way; main is the new way (and the default)
- Prefer to have all the same; most currently master
- Decision: leave it as is (expect to be mixed)
BESS-UPF Helm Charts: Current charts assume symmetry for the access and core interfaces (i.e., same CNI Plug-in for both) Gabriel Arrobo
- Any objection about making the Helm Charts as generic as possible such that each individual CNI can be independently chosen?
- Example: access=macvlan, core=host-device
- But it remains the default that they are the same (currently macvlan); so remains backward compatible
- Will eventually want to expose in OnRamp
Road map cleanup Ajay Lotan Thakur Gabriel Arrobo
- Priority to take up only SBI upgrade this quarter/end of Sept 2024
- Priority on bug fixes and issues reported by other community members
- See Roadmap for more details
- Work harder to get info to the community / work to attract contributors
SD-RAN update Woojoong Kim (Deactivated)
- Working with a team of 5 students
- Github action → Almost done (run tests and publish images)
- Scalable RAN Simulator → on going; hopefully done in Jun.
- Dependency update → dependabot has been busy; resolving 1-by-1 (waiting on GitHub Actions)
OnRamp Update Bilal Saleem
- Separate Management Node → Dealing with local charts was the complication
- Dueling Docker installs → Ansible playbook now checks for existing Docker, per Docker best practices
- Check that multiple gNBs work for data plane
- Working on proposal for incorporating scalable Core into OnRamp
QA Update Kunal Kapoor
- JJB code commit is done
- Working on updating Readme and streamlining the code as per the current requirement.
- David Ferguson (Deactivated) : Customizing the job as opposed to customizing the build machine (so can dynamically grab and release VMs)

6-11-24

LF ticketing systems (see above)
CLA issues: User does not receive confirmation email to complete process (https://github.com/omec-project/pfcpsim/pull/84#issuecomment-2113633448). Additionally, CLA message points to a link that does not exist anymore (https://wiki.opennetworking.org/x/BgCUI) [Gabriel,Woojoong]
- Woojoong: I think before it is fixed, I prefer to disable it till it's ready for SD-RAN/ONOS project.
- Ticket filed with LF.
SD-Core repos: Standardize default branch name ("master" or "main") [Gabriel]
- Lets standardize on master
SD-Core/Aether roadmap
- Captured process - comments welcome
- Current list of feature wishlist captured in roadmap - comments welcome
- See SD-Core Roadmap work items
Jira access - any place its documented?
- Should document and link in the Wiki
- What do we document
  - Still have ONF Jira (at least in name); have migrated hosting VM to LF
  - SD-Core and SD-RAN are two projects under that Jira
  - No obvious legacy value
  - Proposal is to start fresh with Jira
OnRamp Improvements [Bilal]
- Support for separate Mgmt server
- Conflict if Docker already installed
- Propose a blueprint for scalable Core
JJB Status
- Jenkins job creation/update is now handled in the seed job
- To document the process in README of aether-jenkins repo

6-4-24

Archive "tiny" (unused) repos [Gabriel]
- As part of release 1.4, several repos were consolidated into the "util" repo
Do a patch release for Docker images (1.4.1) [Gabriel]
- New Helm Chart also just published
- Expect a PR to update OnRamp
Working on a blueprint for a DPDK configuration
SD-Core Roadmap Discussion
- What's the process
  - We have an old Google Doc
  - Move it in the Wiki
  - Community contributes
- How do we move past Release 15?
  - Start with OpenAPI spec for R17
  - Generate code (and test/fix)
  - Results in the wish-list
SD-RAN Status
- Training new contributors
  - To document in a new wiki page for future onboarding
- Jenkins to Github action conversion for SD-RAN is mostly done
  - Expect to complete this week
ROC Status
- PR now merged
- New Helm Chart published by GitHub Action
- New version (2.1.36) was published => it did not exist (but has been used by OnRamp)
- What gives???
OnRamp Status
- Bilal looking into two issues
- Docker issue when deploying in CloudLab
- Local helm charts on master node
JJB Status
- Testing on AWS (Goal is to be re-producible)
- Automate Jenkins job creation using JJB and definitions from git repo
  - Seed Job on builtin node to pull the latest changes in aether-jenkins repo - Done.
  - Cron job to update the jobs in Jenkins.
...

5-28-24

Monitoring for Aether Registry Failures
- Must renew certs every 3 months
- Auto-renew often isn't so automatic (needs manual intervention)
- Logs can trigger Slack messages → to an LF channel and #aether-dev
- Double check on the 3-month requirement
- Double check on having multiple eyes on the LF channel
Celestic Use Case (Introductions)
- Working with MosoLab on the RAN side
- Small server → Silicom (upper range Intel Atom); need to limit power
- Mongo DB can't start because no AVX (current issue)
  - For 4G, should be able to go back to Mongo v4.4.6
  - Can override with local Helm Chart
- Customers are NGOs
- Starting with LTE (lower cost); will move
ROC Status → Waiting on secrets for the GitHub Actions
SD-RAN Status → Merged PRs to improve Actions
Demo JJB test case → Shoot for one e2e working example
- Need a "test Jenkins" server
- Deliverable: Anyone can clone aether-jenkins repo and reproduce the test (document in the repo's README)
QA Status
- Multi-UPF tests are failing as the second UPF is getting crashed → in discussion with Gabriel.
  - When >1 UPF, we are getting latest UPF version (rather than default)
- UERANSIM throughput test: In progress → Networking issue (setting up tunnel)
  - Test both TCP and UPD traffic
  - Test "output" should show the recorded numbers (on Console)
- Exploring aether-ci-management repo to automate the new onramp integration test JJBs
  - See above.
OnRamp w/ local Charts
- OnRamp runs on a different node than Aether master node
- Bilal uses this config too, but does not local helm charts → Need the combo of both

5-21-24

Draft Wordpress web site now exists (copied from previous prototype)
- Should go live today
- Need to point DNS at new site
- Waiting for credentials to edit
Default parameters for SD-Core in OnRamp (such as use of latest version of sdcore charts, enable SCTPLB and enableDBStore) [Gabriel]
- Latest version of Charts → Always explicitly identify latest, but manually track next version once stable
- SCTPLB → Yes
- enable DBStore → Yes
ROC Update
- Migrating helm charts to GitHub (with Actions)
- Waiting for daf to set up appropriate secrets
SD-RAN status update
- NOTE: Sorry but Woojoong could not join TST in May as he is working from Korea - time zone is a way different. Will update the status TST meeting note in May
- Trying to merge PRs raised by dependbot - security and functional test, which is a part of release process
- Start investigating GitHub action to migrate it from Jenkins - try to migration done in this release cycle
- Planning to introduce the new RAN simulator design to new members so that they can start writing a code on it in this month
- Overall training sessions are ongoing for the new engineering team members - on track!
  - Done: How to develop sd-ran microserice
  - Done: How to use Aether OnRamp for sd-ran dev
  - Done: Golang / K8s / Helm chart basics
  - Done: Dependency among SD-RAN microservices - important to merge changes
  - Next week: K8s operator pattern / webhook service
  - Next week: Essentials for the new RAN simulator implementation
- On-site training session(s) is scheduled next week (1-2 days)
QA
- JJB files for an onramp integration test are committed on Aether-Jenkins repo.
- Updated the default config files for UERANSIM to map it with the UERANSIM's format.
- Debugging on UPF stuck issue with Gabriel.
  - 4G fix was easy: json → jsonc
  - 5G problem is intermittent
- UERANSIM throughput test: There is a change in ip address allocation, working on incorporating it.

5-14-24

New public calendar with all Aether meetings: https://zoom-lfx.platform.linuxfoundation.org/meetings/aether-fund
- Update WIki/Web to point to new calendar
- Prototype web page migrated to WordPress; live soon.
QA
- Making VM-prep explicit (daf)
- Facing an intermittent issue where UPF is getting stuck in terminating state.
  - Kunal to post link to log files to #sdcore-dev
- Github actions for onramp integration tests.
  - Not on the immediate todo list.
  - Short-term goal use JJB defs so job config is visible & editable in GitHub
- Throughput test with UERANSIM discussion
  - Goal is to show best practices in how to measure throughput; don't really care about the AWS numbers, per se.
Have moved ROC Helm Charts from Gerrit to GitHub
- Draft GitHub Actions in place
- Need help from daf with secrets
Presentation on PTF: line-rate tests of UPF
- Link to slide deck: <todo>
- LInk to documentation: https://github.com/omec-project/upf/blob/master/ptf/README.md

5-7-24

RIC Forum SMaRT-5G Demo (Presented end of March)
Progress migrating JJB jobs to GitHub
- Blocker: Reproducing workable VM (starting from "clean" VM)
  - Mark Beierl has had issues with cloud-init in ci, might help
- Blocker: Setting up "structure" on GitHub → Start with one example and find next blocker
Progress migrating to GitHub Actions
- SD-Core → Fully migrated Jenkins CI jobs to Actions
- SD-RAN → First step completed (whatever "make test" did); but "publishing" is still-to-do
- ROC → 5 PRs open to replicated "first step"; need some help from ROC maintainers
Progress with Aether web site
- Should have HTTP redirect up soon (today?)
- Planning a new/permanent site with LF's help
QA
- Debugging ngsetup failure with UERANSIM ( single server configuration) - in progress
  - 2-server config works and is tested by Jenkins integration test
  - One of the solutions to this issue can be adding another virtual interface using Macvlan for gNodeB.
- Working with internal IT team to get the VPN connection with AWS back up.

4-30-24

Conflicts with LF's ONE Summit, call proceeded with Gabriel Arrobo as chair
Gabriel Arrobo mentioned that PTF is ready and linerate tests can proceed, allowing for review of UPF #781
Gabriel Arrobo and Mark Beierl discussed having a demo of PTF and linerate tests in an upcoming TST meeting

Meeting ended early ( ~ 13:10 EDT)

4-23-24

Plans for next week's ONE Summit
- Larry & Timon attending
SD-Core Status Update
- Created a new tags for a source (for Canonical)
- Only 2 open PRs (related to UPF)
- gnbsim doc (including pointers); single source for gbnsim docs
SD-RAN Status Update
- Woojoong unable to attend; nothing to report
OnRamp Status Update
- Improving support for developer workflow
- Merged PR to update to 1.0.1 SD-Core charts (maintenance release)
QA
- Improvement in UERANSIM deployment like single target and code for removal of existing UERANSIM repo
  - When cloning, give it a specific name (perhaps with date) so when you delete, you delete that copy.
- Move job definitions to GitHub - in progress
  - Created aether-onramp jobs definition for 6 jobs
  - Sync with DAF about moving from Gerrit to GitHub
DAF action needed to support GItHub Actions (reminder)
Aether web site should go live soon
- Waiting for LF to update DNS
- Make sure we have "open source 5G" (and similar keywords) for SEO purposes
- Get SEO help
- staging site (feedback from community encouraged)
  - https://aether-project.square.site/

4-16-24

LF Transition Status
- Transferring DNS to LF has caused disruptions; so far easily correctly
- Announcement to old ONF mailing lists to move to new mailing lists?
- Looking into "mailchimp" like facility to do announcements (no LFID required)
- Need a place for new people (info@opennetworking.org) to ask
- info@aetherproject.org works, and goes to larry, michelle, Timon
  - need to put this on the public website
- project website needed
  - LF will link
  - Larry prototyped a site (using Larry's personal webbly account). Need to launch.
    - TODO - get DNS to point to this site
SD-RAN update
- Three grad. students from Korea will work as an engineering team member and 2-3 more will come
  - They are interested with the Scalable RAN Simulator as they are trying to do the similar thing in their lab
  - They have an experience on RICs not only SD-RAN but also Flex and O-RAN SC RIC too
  - They also want to commit their research results to the upstream as a SD-RAN xApp/rApp
- Onboarding plan for new people
  - Woojoong has a plan to visit Korea in May. Will meet them F2F and do some hand-on session so that they can immediately start working for SD-RAN
- Next release is planned for June
OnRamp Status
- Now uses v1.4 of SD-Core (candidate branch now merged into master)
- Were seeing some instability with integration tests on Jenkins
  - Now seem to be fixed
  - Stale info was causing problems
- Several clean-up activities underway
  - Eliminated references to registry.opennetworking.org
  - Removing unnecessary helm repos
  - ...
- Will make 2.2.0 official when we are ready to publicly announce
- How to find the right slack channel → aether-onramp vs aether-dev ??
  - Update channel headers → Tell people the kinds of questions to ask (and where else to go)
SD-Core & OnRamp announcements
- Tied to above bullet about Aether web site.
UPF, DNN, Device Group Question
- Current open issue: https://github.com/omec-project/smf/issues/227
- DNN is being used as the primary key in SMF
- It is possible at the moment to represent an invalid state
  - What direction should be taken to ensure data integrity / How should this be prevented?
- Ajay looking at issue. Canonical will help fix.
QA
- All the integration test jobs are fixed
- Single server UERANSIM blueprint - in progress
  - 2-server config test is successful.
  - Interest in reporting throughput test results (around 800Mbps)
  - UERANSIM blueprint documentation needs to "match" what Jenkins tests
- Next big task: move job definitions to GitHub

4-09-24

Status of SD-Core / OnRamp release
- Version 1.0.0 of SD-Core Helm Charts published (called v1.4 of SD-Core)
- Candidate branch of OnRamp 2.2.0 passing integration tests (plan to merge later today)
- Release Announcement in progress
Registries: opennetworking.org is a mirror, but is referenced by OnRamp (to be fixed before "official" release)
- Should use aetherproject.org registry
SD-RAN Update
- Recruiting PhD students to work on SD-RAN
- Sync'ing up with Gabriel an use of GitHub actions (waiting on DAF)
- Reminder: Take a look at SD-RAN wiki pages
Is it time to form Working Groups?
- Perhaps: SD-Core, SD-RAN, Trial Deployments (or Use Cases)
Proposal: Move Aether-related docs to GitHub pages (freeing up AWS server that is currently being used)
- Minimally, need to move sd-core/sd-ran docs to aetherproject.org (currently opennetworking.org)
- Bring gnbsim docs to the set of Aether docs
https://wiki.onosproject.org/ is down again?

4-02-24

SD-Core / OnRamp release status
- New images, Helm Charts, docs published
  - Automated through GitHub actions
  - No CI dependencies on Jenkins (SD-Core is done)
  - Also applying to other components (see below)
- Candidate branch of OnRamp now testing
  - QuickStart blueprint passes integration test
  - Other blueprints still being tested
- Announcement Plans
  - Obvious new channels → email, slack
  - Can use ONF channels, but should be pushing people to the new channels
  - Timon and Larry to work up a broader plan
Roadmap for SD-RAN & upcoming v1.5 release
- (See Wiki for more info)
- Roadmap page and v1.5 release pages (proposal stage) now on Wiki
- High-level goals
  - Catch up with latest Service Models, etc.
  - Grow the SD-RAN community
    - Recruiting PhD students
    - Possibly engineers with companies contributing to SMaRT-5G
  - Stablize new RANSIM
- SD-RAN Release Process → Moving from Jenkins to GitHub Actions
  - Testing, creating images, unit tests, check license done
  - What's missing is the publishing-related actions
  - Also many backlogged issues
CI shift (Jenkins → Actions) ROC is in progress → Same done for ROC, except PRs are still pending (Jenkins issue)
Action for inactive PRs and issues in repositories (Gabriel)
- https://docs.github.com/en/actions/managing-issues-and-pull-requests/closing-inactive-issues
- https://fvsch.com/stale-bots
- Ongoing vs One-time??
  - 140 total, but only 15 are related to 5gc
  - Other 125 issues are related to 4gc
- Discussion: Interest in maintaining 4G variant of SD-Core
  - Don't archive it (people do use it)
  - No one actively contributing
  - 125 open issues (plus other bug fixes not yet submitted)
  - Add "we are not actively maintaining 4g" comment and close the issues
  - NOT archiving, so someone can pick it up and run
  - Still works as an OnRamp blueprint (using existing images)
  - SD-RAN still works with 4G (using existing images)
- What about stale-bots for 5gc?
  - Currently there for 30 days; increase for 120 days
Registry locations (aetherproject, DockerHub, opennetworking) (Gabriel)
- opennetworking registry seems to mirror DockerHub
- Do we need to still publish images in opennetworking? What is the future for the opennetworking domain?
- Wean ourselves off of opennetworking (both the registry and the DNS name)
- General Note: Look for opennetworking.org usage and remove (check Docs!)
Status of UERANSIM Blueprint / QA Status
- Changes in aether-ueransim repo - PR Raised
- Changes in aether-onramp repo for the UERANSIM integration - PR Raised
- Larry to review

3-26-24

Aether Mailing Lists now up-and-running
- Subscribe at:
- Migration Plan
  - Email to ONF lists inviting people to move
  - Post to Slack inviting people to join lists (but will likely continue using Slack too, where we DO have history)
  - Auto-responses for future inquiries → Waiting so can fix once for all three projects (Aether, P4, Broadband)
Status of SD-Core Release
- Source code is ready for release
- Automating tagging process (waiting on the ONF bot)
- Replicating the VERSION approach used elsewhere in Aether → AMF is "demo" repo
- Immediate goal is to "fix" Jenkins job to publish Helm Charts from GitHub rather than Gerrit
- Long-term goal is to write a GitHub Action to auto-publish Helm Charts (replacing Jenkins)
- With Respect to OnRamp
  - Test "candidate" that uses the latest charts → Define a "candidate blueprint" and run nightly integration tests against it.
  - Upgrade OnRamp from 2.2.0-dev to 2.2.0 (previous was 2.1.0).
- Announcement?
  - Message to announce mailing list (also Slack)
  - SD-Core Docs include Release Notes
Status of SD-RAN Release
Status of UERANSIM Blueprint / QA Status
- 2-Server configuration is the expected default
  - 1-Server config is still problematic – data loop
  - To debug as Future Work
- Changes in aether-ueransim repo - Done
- Changes in aether-onramp repo for the UERANSIM integration - in progress
- Nightly integration test added to Jenkins - in progress
A limitation of OnRamp is that running two Cores (related to how network is configured) can get confused. Include Bilal in a discussion of how to address.

3-19-24

LF Transition Update
- Mailing lists now up: tst@, dev@, announce@lists.aetherproject.org
- Instructions for joining lists: Visit https://lists.aetherproject.org/g/TST (dev | announce) to join
QA Status
- Integrating UERANSIM; expect a PR tomorrow.
Canonical’s Juju-managed Aether 5G Core (Mark Beierl and Serdar Vural)
For lab work using Aether → Please add "abstract/summary" + related links to Research page on wiki.

3-12-24

LF Transition Update
- Wiki → Live at wiki.aetherproject.org
- Mailing Lists → info@aetherproject.org set up (not a list); others (tst, dev, announce) still in progress
- marketing:
  - Can set up Linkedin page? Done for VOLTHA. Marketing in general is addressed by the Board
  - Aether board has to decide on how to apply budget for marketing
- Other board funding opportunities:
  - Intern projects (could be budgeted by the board); also funded internships
    - unfunded internships? In return, certificate of completion??
Asking for help
- Is this a discussion that should be started in the new email forum, or is Slack better for this type of discussion?
- Slack “loses” history, email lists typically have searchable archives
- Once the mailing list is established, Mark Beierl will kick off the conversation there.
UPF Performance Testing
- Mark Beierl wants to look into trex and UPF performance testing
  - At one point ONF used "Packet Testing Framework" to stress test UPF (t-rex generates load)
  - Able to directly test UPF without any other components affecting the performance results (Intel Labs doing this manually)
  - Traffic Generator Intel Lab uses: https://github.com/gab-arrobo/upf/tree/traffic-generator
Request slot for presentation in 2024-03-19 TST meeting for Canonical’s Charmed Aether 5G Core (Mark Beierl and Serdar Vural)
- Shooting for 20 minutes next week.
SD-RAN update (Woojoong)
- Release plan -for v1.4
  - Targeting Q2
  - Woojoong is writing a plan (patterned after the plan for SD-Core)
  - Defining scope
    - Major change is to use O-RAN Service Models
    - Hope to include first version of new scalable RAN-Sim
    - Alex notes the current ASN.1 encoder might not be O-RAN compliant
    - See OAI integration note below
- Scalable RAN-Simulator - getting delayed
- 5G OAI integration?
  - Last Pronto demo: OAI integrated with KPI xApp. (OAI 2022-w52)
    - needs to be released from Intel internal github repositories. Would like to get in 1.4 if possible, but may be delayed
  - Intel Labs has separate OAI integration (OAI 2023-w27)
  - The Intel Lab changes should be merged into OAI official release; may be able to use that in 1.4 release of SD-RAN
QA
- UERANSIM integration
  - Jenkins job for non containerized UERANSIM integration with iperf
  - Link - https://jenkins.aetherproject.org/view/Aether%20OnRamp/job/AetherOnRamp_2servers_20.04_default-charts_UERANSIM_non-dockerised/

3-5-24

LF Transition
- Wiki → Pending
- CLA
- AWS
- Next Steps:
- dev@aetherproject.org → developers across components (open for anyone to join, free to add subgroups as usage dictates, but opt for a common channel for now)
- tst@aetherproject.org → community topics / agenda / Open and public
- announce@aetherproject.org → broadcast announcements
- support@aetherproject.org → ask for help/info, inquiries

Status
Set up project mailing lists; proposal:

Use Cases / Trial Deployments
- bubbleRAN (OAI commercialized stack)
- Purdue → Modified SD-Core to demo scalability of CP
- Princeton → Demo’ing how to monitor RAN activity
- Intel Labs → Security & Slicing (that it used OAI is an artifact that can ber feedback into a blueprint for others to benefit from
- Larry will set up skeleton for this on wiki (once wiki is up)
- Others will flesh out the details.

SMaRT-5G → Power, deployed at WINLAB
ESNET → Science field trial in Colorado
IN-MAC → Indy, Purdue/IN initiative
Research Platform → Purdue, Princeton, Intel Labs,...
Kajeet? 4G deployment
GSLabs + monogoto
Next Step

Release Plans
- In good shape; whittling away at PRs.
- More than 200 PRs open (e.g., dependency updates)
- Have v1.5 release that was ready at one point.
- Activity around RANSIM that’s related
- Who!!!!!!
- Action: Talk to Woojoong and see if he has a proposal
- Questions: Archive SD-RAN RIC in favor of OSC RIC (or Flex RIC)
- Implies a set of “official” blueprints (which implies a set of Helm Chart

SD-Core
SD-RAN
Aether 2.2.0

QA Update
- Few test cases are under testing phase
- Currently local (need to move to public Jenkins)
- Proposal : deploy a non-containerized UERANSIM on node2
- Seems to be working ok (not tested on a single node, co-located with Core)
- To create a Jenkins job to test (and demo)

gnbsim test framework
UERANSIM integration

2-27-24

LF Transition Update (daf)
- Impacts wiki too.
- Certificate has been reset (this is a DNS issue)
- Has no effect on services working

registry.aetherproject.org DNS resolution should be fixed sometime today.
AWS org had been transitioned as of Friday, but not today (looking into)

Guillaume to present proposal for tagging releases

Issue: versioning source for each NF vs versioning SD-Core in aggregate (including helm chart and container artifacts)
Also beyond SD-Core to include all of Aether.

Discussion about the process for tagging auxiliary repos (Gabriel)

Ghislain comment/suggestion: https://onf-community.slack.com/archives/C02TGUWTC93/p1708705492733279
Yes to methodology Guillaume proposes / No need to stay in sync.
Historically tried to pin versions of dependencies (so builds repeatable)
No tag on every commit (had to change VERSION file), but the higher level module pins on a specific version, so if you want to incorporate an update, you need to change the VERSION. Developer judgment on when to update VERSION. CI mechanism does the right thing when the VERSION file changed.

DPDK Issues in the UPF (Mark B.) Background:

VM with 3 virtual interfaces, 16g ram/2x1g HP, 8 cpus
Setup dpdk on host for PCI ids 00:04 and 00:05 (2 of the virtual interfaces)
Start bessd, get error when running up4 script:
EBUSY: Device or resource busy
Followup pointers from Gabriel, Dan, and Larry

Changing bitrate does not affect existing connections.

AI: Mark to open an issue.

QA
- Added validations in the gnbsim integration test.
- New framework : 33 out of 57 TCs are done (Robot-based)
- Still seeing the issue (may be multi-NF version issue)
- Updated webui container works better; release note has been posted; waiting for release instead of back-patching.
- Runs in a container (apart from SD-Core/k8s)
- Has a single interface for both control and data planes

Create a more exhaustive tests for gnbsim : in progress
First UE failure issue
UERANSIM data interface connection

2-20-24

Attending:

Timon Sloane (Aether ED)
Gabriel Arrobo (Intel)
Kunal Kapoor (GS Lab)
Mark Beierl (Canonical)
Ghislain Bourgeois (Canonical)
Guillaume Belanger (Canonical)
Serder Vural (Canonical)
Ajay Lotan Thakur(Intel)
David Ferguson (LF)

Larry unable to attend today

CI/CD pipeline update
- Sctplb images need to push to dockerhub/aether-registry
- Incremental images pushed to aether registry
- Only releases pushed to docker hub
- PR opened but few things need to be fixed.
- Pending: Website publish pending
- Pending: once github works then we can archive gerrit/opencord project for documentation
- Github action ready

Docker images were pushed via jenkins job.
Proposed change:
Documentation Job
Helm Chart Publication

QA
- First basic test is created here
- Create a new framework for onramp
- Recently seen a failure with new image as well

Create a more exhaustive tests for gnbsim : in progress
UE failures

AoB

Is there a TST next week (MWC week)?
General consensus is yes for the meeting

Release planning taking place (3:30PT Thur meeting)

Gabriel, Ajay, DAF
Guillaume would like to join (daf sending invite)
Defining new release process
Doc tracking what is going into the release: https://docs.google.com/document/d/13Y1pSpo6xggRBpfYgQhhavy2eQA4qFj3HFngMmfRX34/edit#heading=h.4ab0vk1erozx

2-13-24

Attending:

Ghislain Bourgeois (Canonical)
Timon Sloane (Aether ED)
Larry Peterson (Princeton)
Gabriel Arrobo (Intel)
Woojoong Kim (Intel)
Andy Bavier (Intel)
Kunal Kapoor (GS Lab)
David Ferguson (ONF)
Scott Baker (Intel)
Ajay Lotan Thakur(Intel)
Mark Beierl (Canonical)

Progress moving to LF tooling (daf/Michelle)
- Includes weekly TST meetings, upcoming events
- Domain is with LF, not associated with Aether project yet – SUPPORT-24447

Create an INDIVIDUAL account
New public Aether calendar: https://calendar.google.com/calendar/u/0?cid=Y18wYzJmYjBkZDE1ZGY3ODk0Y2JlMTYzYWYxYmVjZDJmYTk2OWNkYjhlYjg3NTk0NDdhYjFkMzUwZmE4ZTVkNWYwQGdyb3VwLmNhbGVuZGFyLmdvb2dsZS5jb20
New meeting invite starting next week - will be sent out to mailing lists; can also self-register to attend with this link: https://zoom-lfx.platform.linuxfoundation.org/meeting/92250483404?password=3b574bec-082b-4f50-8368-2c0c290de813
DNS transfers complete => Now can set up wiki (watch for wiki.aetherproject.org)

Update testing framework from SD-Core documentation (Gabriel/Ajay)

Tests on PR failing
https://docs.sd-core.opennetworking.org/master/testing/about_system_tests.html
NG4T is not used anymore
See updated testing section in Aether Guide: https://docs.aetherproject.org/master/testing/about_system_tests.html

Aether Docs moved to GitHub (llp)
- Option for an Aether web page?

https://github.com/opennetworkinglab/aether-docs
Should be able to replicate for other docs
Gitpages are an option for publishing (vs a web server running in AWS)

Preliminary notes on SD-Core Roadmap

https://docs.google.com/document/d/13Y1pSpo6xggRBpfYgQhhavy2eQA4qFj3HFngMmfRX34/edit

Roadmap for Aether OnRamp

https://github.com/opennetworkinglab/aether-onramp/wiki/Roadmap

QA Status
- https://jenkins.aetherproject.org/view/Aether%20OnRamp/job/AetherOnRamp_Quickstart_20.04_custom-charts/
- It seems to solve the timing problem; currently hard coded as a “one time” test -- only need to run until the next release.
- Stay with JJB/Jenkins for this.
- Script that create templates

Integration test with new webui image - Done
JJB/Github actions for the integration test - in discussion with daf
Start building a “exhaustive profile test” (see Core Roadmap)

2-6-24

Attending:

Larry Peterson (Princeton)
Mark Beierl (Canonical)
Timon Sloane (ONF)
Ghislain Bourgeois (Canonical)
Guillaume Belanger (Canonical)
David Ferguson (LF)
Ajay Lotan Thakur (Intel)
Gabriel Arrobo (Intel)
Bilal Saleem ( Purdue)
Andy Bavier (Intel)
Badhrinath Padmanabhan (Intel)
Kunal Kapoor (GS Lab)

Progress scaling SD-Core (Bilal)
Who are the SD-RAN contributors (approving PRs, etc) (Gabriel)

Woojoong → Can he help us grow this community?
Sarat - could he help grow this community? PI on NTIA grant
Alex -- CC when responding to requests
Rimedo ??

SD-Core next release (Gabriel)
- Set of Milestones
- Cadence → 6 months?
- Readiness
- Wait on wiki (can record in Google Docs today)
- Look at during next week’s TST call

Define date / Target set of features/capability to be included
Need to define a Roadmap
LF has managers and processes for releases… need to tap into
Support from daf

sdcore-helm-charts and sdcode-docs are in GitHub (Gabriel)

Need help to make the necessary changes in the automation pipeline to point to these repos instead of the ones in Gerrit
See Aether docs (related)

FOSSA code-scanning issues

Stay with approach that has token in the action

Progress moving aether-docs from Gerrit to GitHub (llp)

Replicates what use to happen on Jenkins as a GitHub Action (rather than pointing Jenkins at GitHub instead of Gerrit… the other approach)

Moving future TST meetings to LF tools (llp)

Watch slack tst channel for updates…
Watch this page for updates…
Watch for a new calendar event…

SD-Core wiki page down which causes the unit tests to fail in link check (issue observed while raising PR on sdcore-docs repo) (Suresh) https://wiki.opennetworking.org/display/COM/SD-Core

ONF Wiki back up – system got corrupted, needed to be rebooted, resulted in changed IP so had to reset the DNS settings this morning.
Other option is to mark onf wiki as an “ignore” in conf.py (until we update docs with links to Aether wiki).

Aether-QA
- Add a new integration test that “hardwires” in latest web console image.
- Issue is tunneling through docker container
- Maybe look at gnbsim; how does it set up tunnel? Keeps control and user plane separate, whereas ueransim uses a single tunnel/interface.
- Look to others that have set up UERANAIM in a separate server.

First ue failure → Need to update image we deploy (update helm charts in progress).
Iperf3 integration to ueransim
ROC re-deployment → No new developments to Aether-Jenkins repo on GitHub (include aether-ci-management “structure” in aether-jenkins).