W1CDN/ham-radio-licenses
1
0
Fork 0
Code Issues 132 Pull Requests Packages Projects Releases Wiki Activity

Don't remove duplicate rows from ARRL scraper.

Browse Source 
Something going on here where the numbers change but the date on the page doesn't.
This commit is contained in:
mattbk 2024-04-21 14:28:34 -05:00
parent c7a2fbddb8
commit 1f9cfb8f9c
1 changed files with 7 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
bin/scrape-arrl-fcc.R
View File

@ -40,11 +40,11 @@ write.table(tab, file = "out/arrl-fcc-licenses-scraped.csv", sep = ",",
na = "")
# Clean up table to remove any duplicates (sometimes the page isn't updated regularly)
db <- read.csv("out/arrl-fcc-licenses-scraped.csv")
db2 <- db %>% distinct(.keep_all = TRUE) %>%
filter(Date != "Date")
write.csv(db2, "out/arrl-fcc-licenses-scraped.csv",
quote = F,
row.names = F,
na = "")
# db <- read.csv("out/arrl-fcc-licenses-scraped.csv")
# db2 <- db %>% distinct(.keep_all = TRUE) %>%
# filter(Date != "Date")
# write.csv(db2, "out/arrl-fcc-licenses-scraped.csv",
# quote = F,
# row.names = F,
# na = "")