DataONE

@dataone

DataONE

Public team

Joined on Nov 4, 2020

  • CN Apache config and Logging CN Apache configuration is (perhaps unnecessarily) complex. /etc/apache2/apache2.conf -> error.log -> conf.d -> event_log-proxy -> rewrite.log -> rewrite_cnquerylog -> rewrite.log -> conf-enabled -> charset.conf -> event_log_proxy.conf
     Like  Bookmark
  • Approaches to cross-repository dataset replication and linking Hackmd.io Link Time: 2021-Nov-10 17:00 UTC 2021-Nov-10 12:00 America/New_York 2021-Nov-10 10:00 America/Denver 2021-Nov-10 09:00 US/Pacific
     Like  Bookmark
  • The indexing process takes existing content, parses it, extracts values to populate fields of an index document, and updates the Solr index with the index document. Indexing performance on the CNs is very slow. A complete re-index currently will take several days, perhaps weeks. The goal is to complete re-indexing of the entire corpus in an hour. One hour is 3600 seconds. There are currently about 3E6 documents in DataONE. Say 3.6E6. That means 1,000 docs per second, on average. Feeding the indexer In DataONE CNs, the indexer running as index_task_processor reads tasks from a postgres database. Those tasks are added to the database by index_task_generator, which listens to events on the Hazelcast system metadata map. The Hazelcast system metadata map emits a large number of events, many of which are not relevant to the indexing process. hzSysMeta -> index_task_generator: change
     Like  Bookmark
  • #monitor #java #jmx On the client: jconsole "service:jmx:rmi://127.0.0.1:9011/jndi/rmi://127.0.0.1:9010/jmxrmi" On the server: -Dcom.sun.management.jmxremote.port=9010 -Dcom.sun.management.jmxremote.rmi.port=9011
     Like  Bookmark
  • Goal is to provide notification of interesting events (record creation, record deletion) and enable subscription to those events. Options There are a couple options for doing this in postgres, though each fundamentally rely on a trigger to invoke some action that sends a message: actor user user -> table: insert table -> messenger: trigger messenger -> subscriber: message
     Like 1 Bookmark
  • Topic: Data Licensing - a Discussion of Alternative Licenses Time: UTC: 2021-05-06, 17:00 Other time zones: Auckland (New Zealand): 05:00 (05-07) Sydney (Australia): 03:00 (05-07) Shanghai (China): 01:00 (05-07)
     Like  Bookmark
  • Topic: Science on Schema.org Guidelines and Experiences Time: 2021-04-01 at 8pm Eastern US time. Other time zones: US/Eastern 2021-04-01 20:00:00 -0400 US/Central 2021-04-01 19:00:00 -0500 US/Mountain 2021-04-01 18:00:00 -0600 US/Pacific 2021-04-01 17:00:00 -0700 US/Alaska 2021-04-01 16:00:00 -0800
     Like 1 Bookmark
  • Topic: Science on Schema.org Guidelines and Experiences Time: 2021-04-01 at 8pm Eastern US time. Other time zones: US/Eastern 2021-04-01 20:00:00 -0400 US/Central 2021-04-01 19:00:00 -0500 US/Mountain 2021-04-01 18:00:00 -0600 US/Pacific 2021-04-01 17:00:00 -0700
     Like  Bookmark
  • Topic: How can DataONE interact with emerging networks Time Pacific Time (US): 16:00 (02/04/2021) Mountain (US): 17:00 (02/04) Central (US): 18:00 (02/04) Eastern (US): 19:00 (02/04) UTC: 00:00 (02/05) Sydney (Australia): 11:00 (02/05)
     Like  Bookmark
  • 7 Jan 2021, 09:00 Pacific Time Zoom Link https://ucsb.zoom.us/j/94309556242 Participants (please add your name, affiliation, and email) Amber Budden, NCEAS/DataONE, aebudden@nceas.ucsb.edu Karl Benedict, UNM/DataONE Community Board, kbene@unm.edu
     Like  Bookmark
  • 3 Dec 2020, 15:00 Pacific Time Participants (please add your name, affiliation, and email) Amber Budden, NCEAS/DataONE, aebudden@nceas.ucsb.edu Ashley Orehek, University of Tennessee Knoxville, aorehek@vols.utk.edu Donna Scott, NSIDC/CIRES/CU Boulder, dscott@nsidc.org Marjorie Mitchell, UBC Okanagan, marjorie.mitchell@ubc.ca Rob Crystal-Ornelas, Berkeley Lab, rcrystalornelas@lbl.gov Erica Krimmel, iDigBio / Florida State Univ, ekrimmel@fsu.edu
     Like  Bookmark
  • Nov 5, 2020 9am Pacfic time Participants (please add your name, email, and affiliation) Matt Jones, jones@nceas.ucsb.edu, DataONE Amber Budden, aebudden@nceas.ucsb.edu, DataONE Erin McLean, mclean@nceas.ucsb.edu, Arctic Data Center Jasmine Lai, jasminelai@nceas.ucsb.edu, Arctic Data Center Dave Vieglais, vieglais@ku.edu, DataONE Karl Benedict, kbene@unm.edu, DataONE Community Board Co-Chair, UNM
     Like  Bookmark