---
# System prepended metadata

title: Country duplicates during import script

---

# Country duplicates during import script
## Background
Import script downloads all articles in Polish with Polish data (topics and countries) and send them to importer. Importer takes everything in queue and creates data in that dataLanguage. If topics and countries already exist, it will overwrite them only when dataLanguage is English.

## The problems
### 1: Duplicate countries 
Existing countries are right now checked by code and sourceIds, but Polish country codes are translated. 

`example: German, GER vs. Niemcy, NIE`

Country code is not a unique constraint, but importer uses it like that. In the case from the example importer will create new country `Niemcy, Nie`. This will cause duplicates in countries. That problem was reported on stage as translations problem, so we didnt know about it, unil we checked database. 

### 2: New data not translated
Import script downloads old articles, that are pushed to importer, they will never come trough updater (which is downloading all locales versions of data - topics, countries). If one of downloaded polish articles contains some new topic or country, it will be imported only in Polish. Database will have Polish data that will be not translated to other langauges until we receive new article with that topic / country.  

## Solutions (Jamal & Pavel suggestions)
1.  Remove `country code` from importer checks. (solves problem 1, not problem 2)
2.  Importer will check if dataLanguage is english. If it is not, importer will download english version and save it instead. Then it saves locale of received dataLanguage - Polish. (fixes 1, fixes 2 partially - we will have at least english version of the new entity and polish locale)
3. Importer will do the same as updater for countries and topics - upsert englishd data and download all locales. Problem of this is performance of importer. Draft of this solution is here:

https://gitlab.ack.ee/Backend/flash-sport-importer/-/merge_requests/330