W1CDN/ham-radio-licenses
1
0 Fork 0
Code Issues 133 Pull Requests Packages Projects Releases Wiki Activity

ARRL FCC license counts - ongoing #20

New Issue
Open
opened 2024-01-11 09:12:25 -06:00 by W1CDN · 2 comments
W1CDN commented 2024-01-11 09:12:25 -06:00
Owner
Copy link
Copy source

The page at https://www.arrl.org/fcc-license-counts has license counts that appear to update daily.

I have a cron job set up (using https://github.com/overcast07/wayback-machine-spn-scripts) to automatically archive this page daily to the Internet Wayback Machine.

Periodically, these numbers can be manually copied into the CSV file until someone writes a script to scrape the data into a file, probably a separate one that represents machine-collected data that would need to be QCed periodically.

The page at https://www.arrl.org/fcc-license-counts has license counts that appear to update daily. I have a cron job set up (using https://github.com/overcast07/wayback-machine-spn-scripts) to automatically archive this page daily to the Internet Wayback Machine. Periodically, these numbers can be manually copied into the CSV file until someone writes a script to scrape the data into a file, probably a separate one that represents machine-collected data that would need to be QCed periodically.
W1CDN commented 2024-01-19 16:07:24 -06:00
Author
Owner
Copy link
Copy source
# Archive FCC license counts from ARRL
0 9 * * MON-FRI su matt -c "/home/matt/wayback-machine-spn-scripts/spn.sh https://www.arrl.org/fcc-license-counts">/dev/null 2>&

Need to turn into a script and call the script instead.
ham-license-count.sh

#~ /bin/bash
# Archive FCC license counts from ARRL
#0 9 * * MON-FRI su matt -c
/home/matt/wayback-machine-spn-scripts/spn.sh https://www.arrl.org/fcc-license-counts
/home/matt/wayback-machine-spn-scripts/spn.sh https://www.ae7q.com/query/stat/LicenseUSA.php
/home/matt/wayback-machine-spn-scripts/spn.sh https://hamcall.net/hamcallcounts.html

0 9 * * MON-FRI su matt -c "/home/matt/ham-license-count.sh">/dev/null 2>&1

``` # Archive FCC license counts from ARRL 0 9 * * MON-FRI su matt -c "/home/matt/wayback-machine-spn-scripts/spn.sh https://www.arrl.org/fcc-license-counts">/dev/null 2>& ``` Need to turn into a script and call the script instead. `ham-license-count.sh` ``` #~ /bin/bash # Archive FCC license counts from ARRL #0 9 * * MON-FRI su matt -c /home/matt/wayback-machine-spn-scripts/spn.sh https://www.arrl.org/fcc-license-counts /home/matt/wayback-machine-spn-scripts/spn.sh https://www.ae7q.com/query/stat/LicenseUSA.php /home/matt/wayback-machine-spn-scripts/spn.sh https://hamcall.net/hamcallcounts.html ``` `0 9 * * MON-FRI su matt -c "/home/matt/ham-license-count.sh">/dev/null 2>&1`
W1CDN commented 2024-04-07 09:58:58 -05:00
Author
Owner
Copy link
Copy source

Setting up https://amiok.net/gitea/W1CDN/ham-radio-licenses/src/branch/main/bin/scrape-arrl-fcc.R to run daily and drop the table into a CSV. Then I can manually copy the data (once a week or so) into the repo faster than I can go day-by-day through wayback machine.

Setting up https://amiok.net/gitea/W1CDN/ham-radio-licenses/src/branch/main/bin/scrape-arrl-fcc.R to run daily and drop the table into a CSV. Then I can manually copy the data (once a week or so) into the repo faster than I can go day-by-day through wayback machine.
W1CDN added the data-new label 2024-09-16 11:48:47 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Labels
Clear labels
bug
Something is not working
 data-fix
data-new
duplicate
This issue or pull request already exists
 enhancement
New feature
 help wanted
Need some help
 invalid
Something is wrong
 question
More information is needed
 wontfix
This won't be fixed
No labels data-new
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
W1CDN (Matt )
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: W1CDN/ham-radio-licenses#20