---
title: Recommendation system data pipeline
tag: KHTN khtn recommendation system
description:
---
Recommendation system: data pipeline
===

## features - data mapping
- User profile
- User Age: Profile.age
- User Location: User.country
- User Type: Profile.type
- Gender: Profile.gender
- Video profile
- Genre: VideoV2.genre
- Video category: VideoV2.categories
- Video duration: VideoV2.duration
- Video total like: VideoV2.totalLike
- Video total views: VideoV2.totalView
- Playlist id: PlaylistItemV2.playlistID
- Watching behaviors
- like
- valid ping counts: Video_Load_snapshot.valid_pings_count

## Preprocessing steps
- User
- Select users watched at least 10 (5 in code) videos
- Profile
- _isDeleted != True
- VideoV2
- _isDeleted != True and adminBlocked != 1
- categories.replace(r"[\['\]]", "", regex=True)
- PlaylistItemV2
- _isDeleted != True and adminBlocked != 1

## MongoDB
### Provided collections
- VideoV2
- User
- Profile
- Genre
- CategoryV2
- MPAA
- PlaylistItemV2
### Pubsub