Ruck and Rover notes #33

# Ruck and Rover notes #33 ###### tags: `ruck_rover` :::info Important links for ruck rover's [ruck/rover links to help](https://hackmd.io/07z0xroHTFi2IbX93P5ZfQ) **Ruck Rover - Unified Sprint #33** Dates: Sep 9 - Sep 29 Tripleo CI team ruck|rover: Folco/Bhageshris OSP CI team ruck|rover: wznoinsk|ruck, TalmoR|ruck ::: [TOC] --- ## on-going issues :::danger ## TripleO ### gate * Bug: standalone upgrade train failing: https://bugs.launchpad.net/tripleo/+bug/1896595 * Note that job is non-voting so not very high priority. ### periodic / 3rd party - master - Bug: https://bugs.launchpad.net/tripleo/+bug/1897863 (not a promotion blocker) - https://review.opendev.org/#/c/755252/ (Add support for rdo openvswitch layered upgrade special treatment.) - Except standalone upgrade master job all jobs are **GREEN** - train - we are getting NODE_FAILURE on periodic-tripleo-centos-8-train-promote-promoted-components-to-tripleo-ci-testing, waiting for next run. - ussuri - Except standalone upgrade ussuri job all jobs are **GREEN** - stein @bhagyashris @rfolco please focus on stein upgrade jobs. * UPGRADE JOBS voting and failing - undercloud upgrade: https://bugs.launchpad.net/tripleo/+bug/1896248 - standalone upgrade: https://bugs.launchpad.net/tripleo/+bug/1896537 - https://zuul.openstack.org/builds?job_name=tripleo-ci-centos-7-standalone-upgrade-stein - rocky ## OSP ::: --- ## Sept 30th ### TripleO * gate * periodic / 3rd party **master** - Bug: https://bugs.launchpad.net/tripleo/+bug/1897863 (not a promotion blocker) - https://review.opendev.org/#/c/755252/ (Add support for rdo openvswitch layered upgrade special treatment.) - Except standalone upgrade master all jobs are **GREEN** **ussuri** - Except standalone upgrade ussuri job all jobs are **GREEN** **train** - we are getting NODE_FAILURE on periodic-tripleo-centos-8-train-promote-promoted-components-to-tripleo-ci-testing, waiting for next run. * component pipeline: master: https://bugs.launchpad.net/tripleo/+bug/1897947 ### OSP ## Sept 29th ### TripleO * gate * periodic / 3rd party **master** - ~~https://review.opendev.org/#/c/754783/~~ - Note: Jobs will pass, once will get the baremetal component promotion. **ussuri** * fs039 is failing because of tempest test failure - https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp_1supp-featureset039-ussuri/d5b47d0/logs/undercloud/var/log/tempest/stestr_results.html.gz - Note: Will wait for next run as it's not consistent failure. **train** * **GREEN** ### OSP ## Sept 28th ### TripleO * gate * periodic / 3rd party **master** * Bug: https://bugs.launchpad.net/tripleo/+bug/1897505 (overcloud-prep-images failes at Introspect overcloud images (fs001, fs002 and fs039 is failing)) - [DNM] Increase ironic conductor conf values https://review.opendev.org/754690 - https://review.opendev.org/#/c/754783/ - Testproject: https://review.rdoproject.org/r/29351 ``` bhagyashri|rover> hjensas, hi need one help : few CI jobs are failing on master because of https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-master/0ca0cb9/logs/undercloud/var/log/containers/ironic-inspector/ironic-inspector.log.txt.gz <bhagyashri|rover> it's basically failing at overcloud image introspection <bhagyashri|rover> do you have any idea?hoonetorg hrybacki <bhagyashri|rover> hjensas, https://bugs.launchpad.net/tripleo/+bug/1897505 <hjensas> bhagyashri|rover: hm, wonder if it could be that it is slow to transition state in the host cloud? Can you try playing with ironic.conf [conductor]ower_state_change_timeout = 60 and [conductor]sync_power_state_interval = 60 ? Maby bump to 90 or 120 ? ``` **ussuri** * **GREEN** except sc010-ovn - skiplist: https://review.rdoproject.org/r/29709 **train** * **GREEN** **stein** * featureset002-stein-upload failing with (Connection to the 10.0.0.111 via SSH timed out.) - https://logserver.rdoproject.org/openstack-periodic-integration-stable3/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-stein-upload/f2d0f0c/logs/undercloud/home/zuul/tempest/tempest.html.gz ### OSP ## Sept 25th ### TripleO * gate * periodic / 3rd party **master** * **In the current run master is GREEN** * Bug: https://bugs.launchpad.net/tripleo/+bug/1897213 ([inconsistent] scenario010-standalone-master is getting TIMED_OUT at Execute tempest test task) * Fix: https://review.rdoproject.org/r/29634 **ussuri** **train** * Bug: https://bugs.launchpad.net/tripleo/+bug/1897228 * Fix: https://review.opendev.org/#/c/754281/ * Testproject: https://review.rdoproject.org/r/#/c/29252/ ### OSP ## Sept 24th ### TripleO * Bug: https://bugs.launchpad.net/tripleo/+bug/1896917 ( problem with some vexxhost hypervisor) * gate * periodic / 3rd party **master** - periodic-tripleo-ci-centos-8-scenario010-ovn-provider-standalone-master >> https://review.rdoproject.org/r/29619 should fix - periodic-tripleo-ci-centos-8-standalone-full-tempest-scenario-master (will open bug if happens again) **ussuri** - https://bugs.launchpad.net/tripleo/+bug/1895705 scen10-ovn >> skiplist >> https://review.opendev.org/#/c/754198/ **train** - ~~https://bugs.launchpad.net/tripleo/+bug/1895705 scen10-ovn~~ skiplist - https://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-centos-8-ovb-3ctlr_1comp_1supp-featureset039&branch=stable%2Ftrain# known? ~~- <ykarel> rfolco|ruck, https://review.rdoproject.org/r/29617~~ **stein** - several jobs failing tempest tests example: https://logserver.rdoproject.org/openstack-periodic-integration-stable3/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset010-stein/f8673fb/logs/undercloud/home/zuul/tempest/tempest.html.gz (no action taken yet) ### OSP ## Sept 23th ### TripleO * gate * Bug: https://bugs.launchpad.net/tripleo/+bug/1896738 * ~~Fix: https://review.opendev.org/#/c/753564/~~ * Fix: https://review.opendev.org/#/q/topic:linters-fix+(status:open+OR+status:merged) * ~~Most of the failures are because of POST_FAULURES and that was infra issue and that got fixed. https://review.opendev.org/#/c/753498/1~~ * periodic / 3rd party **master** **ussuri** **train** * **GREEN** **stein** ### OSP ## Sept 22th ### TripleO * gate * Bug: https://bugs.launchpad.net/tripleo/+bug/1896595 ([c7 train] ConflictException: 409: Client Error for url: http://192.168.24.1:9696/v2.0/networks.json, Unable to create the flat network. Physical network datacentre is in use failing standalone upgrade train ) * periodic / 3rd party **master** **ussuri** **train** * **GREEN** :) **stein** * Bug: https://bugs.launchpad.net/tripleo/+bug/1896537 * Summary: 1 . It might be similar to https://bugs.launchpad.net/tripleo/+bug/1895822 (standalone-upgrade ussuri) 2. in both cases it passes deployment & upgrade OK but it seems there is something happening during the upgrade that kills connectivity 3. we have a cix on that one so few folks have looked into it but we don't have root cause yet ### OSP ## Sept 21th ### TripleO * gate * ~~Bug: https://bugs.launchpad.net/tripleo/+bug/1896439~~ * ~~Fix: https://review.opendev.org/#/c/752881/~~ * ~~Fix: https://review.opendev.org/#/c/752902/~~ * Bug: https://bugs.launchpad.net/tripleo/+bug/1896469 * Fix: https://review.opendev.org/#/c/752908 * periodic / 3rd party **master** * ~~Bug: https://bugs.launchpad.net/tripleo/+bug/1896469~~ * ~~master pipeline got hit by RETRY_LIMIT because of this change: https://review.rdoproject.org/r/#/c/29009/ (Update rdo-openvswitch to support ovs/ovn2.13 NFV SIG builds)~~ * ~~Fix: https://review.rdoproject.org/r/29542 (Revert "Update rdo-openvswitch to support ovs/ovn2.13 NFV SIG builds")~~ ~~~ <bhagyashri|rover> ykarel, hi master jobs are getting into retry limit https://logserver.rdoproject.org/openstack- periodic-integration-main/opendev.org/openstack/tripleo- ci/master/periodic-tripleo-ci-centos-8-standalone-master/8864213/job-output.txt <bhagyashri|rover> ykarel, looks like this is https://review.rdoproject.org/r/#/c/29009/ hitting retry limit <bhagyashri|rover> amoralej, ^^ <amoralej> bhagyashri|rover, where? <bhagyashri|rover> amoralej, https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-master/8864213/job-output.txt <amoralej> mmm, it's missing nfv repo <amoralej> ykarel, ^ <amoralej> oh, the multinode part <amoralej> grrr <amoralej> but we tested it in multinode, iirc <ykarel> amoralej, bhagyashri|rover looking <ykarel> amoralej, uhh so we tested it but at that point of job master deps repo was used <amoralej> ykarel, let's revert ~~~ **ussuri** * Same issue as master **c8 train** * :green_heart::green_heart::green_heart: **stein** * Bug: https://bugs.launchpad.net/tripleo/+bug/1896220 * Bug: https://bugs.launchpad.net/tripleo/+bug/1896537 ## Sept 18th ### TripleO * gate * [LP: 1896178](https://bugs.launchpad.net/tripleo/+bug/1896178) ( [master][train] nothing provides libmysqlclient.so.21()(64bit) needed by collectd-mysql-5.11.0-2.el8.x86_64) Fix: https://review.opendev.org/#/c/752621/ (Temporarily drop mysql) Note: Please check details on bug comment section. * periodic / 3rd party **master** * Blocker: [LP: 1896178](https://bugs.launchpad.net/tripleo/+bug/1896178) ~~Fix: https://review.rdoproject.org/r/#/c/29472/ (Add temporary hotfix repo for CentOS 8 issue with myslq module)~~ Related patches: * ~~https://review.rdoproject.org/r/#/c/29458/ (Use mariadb connector instead of mysql)~~ * ~~Note: Test patch also passed https://review.rdoproject.org/r/#/c/29227/~~ **ussuri** * **GREEN** :) :green_heart::green_heart::green_heart: **c8 train** * Blocker: [LP: 1896178](https://bugs.launchpad.net/tripleo/+bug/1896178) Same as master one. * ~~test_listener_CRUD (scen10-ovn)~~ * ~~BUG: https://bugs.launchpad.net/tripleo/+bug/1895705~~ (bug still open, tests on skiplist) * ~~skiplist: https://review.opendev.org/751162 [stein|train] Add octavia tests to skip list **(Merged)**~~ **stein** * Bug: https://bugs.launchpad.net/tripleo/+bug/1896220 * fs002, fs010 and fs030 is failing because of Memory issue. It will pass in the next run. - ~~fs037 prepare-parameter:~~ - ~~FIX: https://review.opendev.org/#/c/751340/ **(Merged)**~~ - ~~testproject: https://review.rdoproject.org/r/#/c/29297/ (Passing)~~ - ~~fs038 tempest octavia:~~ - ~~BUG: https://bugs.launchpad.net/tripleo/+bug/1895248~~ - ~~Fix: https://review.opendev.org/#/c/751162/ **(Merged)**~~ ### OSP ## Sept 17th ### TripleO https://bugs.launchpad.net/tripleo/+bug/1896126 >> tempest workspace * gate * ~~https://review.opendev.org/#/c/752187/ (Increase configure_swap_size to 4096)~~ * ~~https://review.opendev.org/#/c/752129/ (Inject both paths for validations roles location) >> ussuri~~ * ~~https://review.opendev.org/#/c/752197/ (Inject both paths for validations roles location) >> train~~ * periodic / 3rd party **master** * ~~https://bugs.launchpad.net/tripleo/+bug/1895792 ( public endpoint for compute service not found is getting TIMED_OUT on fs002, fs020 and fs035 Edit)~~ * ~~fix : https://review.opendev.org/#/c/752199/1 (Run some more featuresets with baremetal provisioning)~~ * ~~https://bugs.launchpad.net/tripleo/+bug/1874418 ((inconsistent) periodic centos8 fs39 fails sometimes with Error, some other host (FA:16:3E:B5:60:2A) already uses address 10.0.0.1)~~ * ~~https://review.opendev.org/#/c/752370/ (Enable undercloud nova for fs020) got +w on it~~. Testproject patch : https://review.rdoproject.org/r/#/c/29351/ (GREEN) * ~~in the recent run build-containers-ubi-8-push job failed here https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-build-containers-ubi-8-push/4f35982/logs/build.log~~ * ~~**Note:** on the testproject patch it passed https://review.rdoproject.org/r/#/c/29227/ , so waiting for next run.~~ **ussuri** * featureset039-ussuri is failing because of this https://bugs.launchpad.net/tripleo/+bug/1874418 ((inconsistent) periodic centos8 fs39 fails sometimes with Error, some other host (FA:16:3E:B5:60:2A) already uses address 10.0.0.1) though this not promotion blocker. **Rest all GREEN** :) **c8 train** * **GREEN** :) ## Sept 16th ### TripleO * gate * https://review.opendev.org/#/c/752187/ (Increase configure_swap_size to 4096) * https://review.opendev.org/#/c/752129/ (Inject both paths for validations roles location) >> ussuri * https://review.opendev.org/#/c/752197/ (Inject both paths for validations roles location) >> train * ~~https://review.opendev.org/#/c/751653/ (Inject both paths for validations roles location)~~ * ~~https://review.opendev.org/#/c/751828/ (Increase configure_swap_size to 4096)~~ * ~~https://review.opendev.org/#/c/751930/ (skip test_snapshot_pattern in master)~~ * ~~https://review.opendev.org/#/c/750967/ (Fix duplicated declaration when the deprecated parameter is used) This will fix sc12 issue~~ * periodic / 3rd party **master** * https://bugs.launchpad.net/tripleo/+bug/1895792 ( public endpoint for compute service not found is getting TIMED_OUT on fs002, fs020 and fs035 Edit) * here is the fix : https://review.opendev.org/#/c/752199/1 (Run some more featuresets with baremetal provisioning) * https://bugs.launchpad.net/tripleo/+bug/1874418 ((inconsistent) periodic centos8 fs39 fails sometimes with Error, some other host (FA:16:3E:B5:60:2A) already uses address 10.0.0.1) ~~* https://bugs.launchpad.net/tripleo/+bug/1895828 ( tempest.lib.exceptions.UnexpectedResponseCode: Unexpected response code received is failing on scenario010-standalone-master)~~ * Note: this issue is not seen on testproject run https://review.rdoproject.org/r/#/c/29351/ will wait for next run - ~~https://bugs.launchpad.net/tripleo/+bug/1895580 'TagFloatingIpTestJSON' object has no attribute 'assertItemsEqual'~~ - ~~https://review.opendev.org/#/c/750369/~~ - ~~https://review.opendev.org/#/c/747760/~~ - ~~https://review.opendev.org/#/c/751244/ (Revert "Change path for validation Ansible files")~~ **ussuri** * https://review.opendev.org/#/c/752187/ (Increase configure_swap_size to 4096) * https://review.opendev.org/#/c/752129/ (Inject both paths for validations roles location) **Note: For ussuri promotion will need above two patches to merge soon.** **c8 train** * ~~privilege escalation (undercloud-containers)~~ * ~~BUG: https://bugs.launchpad.net/tripleo/+bug/1895700 **Note:** In the recent run, didn't see this issue and job is GREEN. Will wait for next run result to reconfirm this issue.~~ * fs039 is in RETRY_LIMIT: Quota exceeded, too many key pairs. ~~https://logserver.rdoproject.org/openstack-periodic-integration-stable2/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp_1supp-featureset039-train/381071f/job-output.txt~~ ~~Note: @weshayutin will need to delete stale ports on rdo-cloud and vexxhost~~ **stein** ### OSP ## Sept 15th ### TripleO * gate * https://bugs.launchpad.net/tripleo/+bug/1895601 (tripleo standalone centos-8 tempest fail: estSnapshotPattern:_run_cleanups): 500 DELETE http://192.168.24.3:9292/v2/images) ~~https://review.opendev.org/#/c/751930/ (skip test_snapshot_pattern in master)~~ * **List of patches needs to be merge to clear gate faiures**: ~~https://review.opendev.org/#/c/751653/ (Inject both paths for validations roles location)~~ ~~https://review.opendev.org/#/c/751828/ (Increase configure_swap_size to 4096)~~ ~~https://review.opendev.org/#/c/751930/ (skip test_snapshot_pattern in master)~~ https://review.opendev.org/#/c/751771/ (Add the variable list_skipped_job) https://review.opendev.org/#/c/750967/ (Fix duplicated declaration when the deprecated parameter is used) * periodic / 3rd party **master** * tempest neutron plugin * ~~BUG: https://bugs.launchpad.net/tripleo/+bug/1895580 ‘TagFloatingIpTestJSON’ object has no attribute ‘assertItemsEqual’~~ **ussuri** * couldn't resolve module/action 'warn'" * BUG: https://bugs.launchpad.net/tripleo/+bug/1895507 * FIX: https://review.opendev.org/#/c/751653/ **c8 train** * test_listener_CRUD (scen10-ovn) * BUG: https://bugs.launchpad.net/tripleo/+bug/1895705 ~~* privilege escalation (undercloud-containers) * BUG: https://bugs.launchpad.net/tripleo/+bug/1895700~~ * memory (fs001, fs002, fs020, fs035, fs039) * FIX: https://review.rdoproject.org/r/#/c/29335/ (Increase configure_swap_size to 4096 for ovb jobs) * Related-Bug: #1895290 * Related-Bug: #1895288 **stein** * container-prepare-parameter.yaml * FIX: https://review.opendev.org/#/c/751340/ (**STILL UNRESOLVED**) * testproject: https://review.rdoproject.org/r/#/c/29297/ ### OSP ## Sept 14th ### TripleO * gate (clear) - ~~https://bugs.launchpad.net/tripleo/+bug/1895027 scen12~~ - ~~https://review.opendev.org/#/c/750980 lower constraints~~ - ~~https://bugs.launchpad.net/tripleo/+bug/1895056~~ * periodic / 3rd party **master** * ~~**NEW!!** https://bugs.launchpad.net/tripleo/+bug/1895580 'TagFloatingIpTestJSON' object has no attribute 'assertItemsEqual'~~ * https://bugs.launchpad.net/tripleo/+bug/1895507 (Standalone jobs failing with "couldn't resolve module/action 'warn'" ) https://review.opendev.org/#/c/751653/ (Inject both paths for validations roles location) https://review.opendev.org/#/c/751766/ ( Revert "Remove objects migrated to validations-common") **c8 train** * https://bugs.launchpad.net/tripleo/+bug/1895290( [train] ERROR root [ ] Image prepare failed: A process in the process pool was terminated abruptly while the future was running or pending failing on few jobs) hare is the fix: https://review.rdoproject.org/r/#/c/29313/ (Increase configure_swap_size to 4096) testing here: https://review.rdoproject.org/r/#/c/28452/ , https://review.rdoproject.org/r/#/c/29254/ ### OSP ## Sept 11th ### TripleO * gate * periodic / 3rd party **master** * master jobs are still failing with https://bugs.launchpad.net/tripleo/+bug/1895056 (package conflict, openstack-tripleo-validations 12.4.1-0.20200909044505.ae20b37.el8.noarch and validations common-1.1.1-0.20200908154415.946e3a8.el8.noarch) https://review.opendev.org/#/c/751244/ (Revert "Change path for validation Ansible files") Hope this will solve the problem **ussuri** * standalone-on-multinode-ipa-ussuri: failed at Execute tempest test: https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-on-multinode-ipa-ussuri/315bb39/job-output.txt https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-on-multinode-ipa-ussuri/33d2ced/logs/undercloud/var/log/tempest/stestr_results.html.gz * scenario010-standalone-ussuri: failed at Execute tempest : https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario010-standalone-ussuri/d843988/logs/undercloud/var/log/tempest/stestr_results.html.gz * **Note:** Ran both the job in testproject : https://review.rdoproject.org/r/#/c/29250/ (standalone-on-multinode-ipa is still failing and scenario010-standalone is passed) **c8 train** * https://bugs.launchpad.net/tripleo/+bug/1895288 ( [train] MemoryError: [tripleotraincentos8/centos-binary-nova-libvirt] Memory Error failing on few jobs) * https://bugs.launchpad.net/tripleo/+bug/1895290 ( [train] ERROR root [ ] Image prepare failed: A process in the process pool was terminated abruptly while the future was running or pending failing on few jobs) **stein** * https://bugs.launchpad.net/tripleo/+bug/1895248 ([stein] (LoadBalancerScenarioTest:test_load_balancer_ipv4_CRUD) show_loadbalancer provisioning_status updated to an invalid state of ERROR is failing on fs038) https://review.opendev.org/#/c/751162/ (Add octavia test to skip list) * fs037 is blocking promotion: https://bugs.launchpad.net/tripleo/+bug/1895314 ### OSP ## Sept 10th ### TripleO (rfolco) cix cards updated, just do a quick pass before monday cix call * gate https://bugs.launchpad.net/tripleo/+bug/1895005 (Tempest failure: cloud overcloud is not found stable/train) https://bugs.launchpad.net/tripleo/+bug/1895138 ( centos-8 standalone-upgrade-ussuri fails build-test-packages issue creating /root/DLRN) marios is working on it. * RDO CI **master** * master promotion has been bolocked because of this issue: https://bugs.launchpad.net/tripleo/+bug/1895056 ( package conflict, openstack-tripleo-validations-12.4.1-0.20200909044505.ae20b37.el8.noarch and validations-common-1.1.1-0.20200908154415.946e3a8.el8.noarch) As below patches got merged and that hits this issue: https://review.opendev.org/#/c/750369/1 https://review.opendev.org/#/c/747760/ ~~~ Tengu> bhagyashri|rover: there's a patch on-going, but it has some issues in CI https://review.opendev.org/713204 ~~~ * recently 'periodic-tripleo-ci-build-containers-ubi-8-push' failed to push the conatiners because of **connection refused** issue but it passed on testproject: https://review.rdoproject.org/r/#/c/29227/ ~~~ noop SUCCESS in 0s periodic-tripleo-ci-build-containers-ubi-8-push SUCCESS in 47m 53s ~~~ **c8 train** : is good today :) - passed all jobs except scenario010-ovn-provider-standalone-train ### OSP ## Sept 9th ### TripleO - gate is bad (rfolco looking) ### OSP

Syntax	Example	Reference
# Header	Header	基本排版
- Unordered List	Unordered List
1. Ordered List	Ordered List
- [ ] Todo List	Todo List
> Blockquote	Blockquote
Bold font	Bold font
Italics font	Italics font
~~Strikethrough~~	~~Strikethrough~~
19^th^	19^th
H~2~O	H₂O
++Inserted text++	Inserted text
==Marked text==	Marked text
[link text](https:// "title")	Link
![image alt](https:// "title")	Image
`Code`	`Code`	在筆記中貼入程式碼
```javascript var i = 0; ```	`var i = 0;`
:smile:		Emoji list
{%youtube youtube_id %}	Externals
$L^aT_eX$	L^aT_eX
:::info This is a alert area. :::	This is a alert area.