## About Me
### Tell Me About Yourself
I am a software engineer with 20 years of experience, and worked in 3 startups. I love software development, building many applications from the ground up, Involved in complete software development life cycle. Plenty of experience from designing, developing, testing and deployment.
I focus on the long-term valuable software solution for the customers with creative ways.
I specialize in Java server-side application development, with hands-on experiences in full-stack development. I am also proficient in several programming languages, such as Go, NodeJS, Ruby. I am proficient in Linux, Docker.
I have great passion for my work.
### What are your strengths?
One of my strengths is curiosity and life learning.I love learning new tech and implementing those. Keeping learning and being curious always refreshes my mind with better designs and solutions.
Friendly with teamwork, my last start-up co-founder, we work as a team for 20 years.
### What are your weaknesses?
I tend to take on the whole project without asking other’s help. In the past, this let me experience the unnecessary pressure and stress.
One specific example was in my previous company, I was co-founder of the company. Because of limited resources. I tried to do everything on my own, including designing, developing both backend and frontend, from building, testing and deployment, and doing support, working closely with the customer.
I was so stressed. This often leads me to overwork and let me feel burned out.
### Why Facebook
Facebook build technologies that help people connect with friends and family, find communities, and grow businesses.
I personally have used many Facebook products and services. Like Facebook website, WhatsApp.
Facebook employees work in small teams that move fast and iterate to develop new products.
I can take ownership of the projects I am truly passionate about. I am sure I can take the challenges and provide successful solutions.
Therefore, this is a place where I can learn a lot, and use my skills and experience to make things happen with a team.
### Why this Position
I am interested in this position because the team tasked with improving user experience in Facebook. I am really excited about learning and using new technologies to develop valueable system, and will be learned a lot from talent team workers.
## Experience
### Linux
I have many years of experience in linux environment. because our products were deployed in linux appliance. so I do some research to how to build iso file for system installation. and write a script to build our product from source code to iso distribution.
Because we have many customers which located in different cities. each customer maybe deploy several machines to run our application. so there were almost 1000 servers. I have done a lot of administration on linux, from Linux server profiling, disk management, application management, upgrade and update system.
to manage the servers well, I also used many linux tools, such as bash, awk, sort, uniq, sed. to reduce manual operation, I often wrote bash script to do automation, batch work.
Because our servers were usually under stress, so cpu, memory, disk, traffic were easily occurred. so to quickly find out the alert, I also built a central monitor system to send a notification when a alert occurred. the monitor system also provide dashboard which display the trend graph for cpu, memory, disk, traffic.
Because our products, I experienced many open source software, such as linux, apache, nginx, haproxy, squid, ssh.
I also find ansible are quite useful. it help us to easily setup a server to install our product. and keep our configuration uptodate.
### Languages
I specialize in Java server side application development. and I am also proficient with several programing language such Go, NodeJS, Ruby, Python.
### Enterprise
Because most of our products are for enterprise customer, so we need to do a lot of integration works. I could give some examples.
such users authentication, in our email security product, it allow users to login in system to check their spam messages. so we need to integrate with current enterprise user system, such as generic LDAP server, Microsoft Active Directory, Microsoft Exchange, even some costomize for database authentication system.
we also provide restful API for users, so they can integrate to their mail server system.
### Docker
Docker is useful, so I use Docker in development, testing, production environment. it help us to keep our environment uptodate.
### Protocols
I had many years of experince working with network protocol.
The first two companies I worked in is about email server and email security. so I am expertise with email protocols, such as SMTP, POP3, IMAP, MIME.
I also implemented a standard filter language: sieve, so we can write rules in Sieve to do filtering.
My third company I worked in is about internet content security, so I am expertise with some more protocols, such HTTP, DNS, SSH, MMS.
### Projects
ground up
softwar development life cycle
full stack
responsibility: support & customer
mail server
email security
internet content security
internet content cache
mms filter
web filter
dsp ad
webrtc video conference
### Infrastructure
Almost all of development infrastructure, such as
I setup redmine for issues tracking and wiki document server.
I setup nagios collectd for monitoring and alert server
I setup license server for our product.
I setup gitlab server for our code repository.
I setup our mail server.
Create libraries and frameworks which will be used across multiple application development projects.
I worked in my previous company, Surfront, for almost 12 years. We developed a lot of applications. many of our applications has a lot of common modules. such as
communication module,
appliance module,
metrics & report modules
so we can easily and fast build similar application from ground up.
### Being a serial opportunist
In my first company, I was involved in the company product development, we developed an email system for a large number of user based customers.
In my second company, I joined a startup company, developed a product from the ground up. I was involved in developing an email security product, which could filter spam/virus/content messages. The product was successful. the company was acquired by a listing company.
Then, My co-worker and I decided to start a new company, which at first focused on internet content security, such as a web filter, which could control whether the website could be accessed by workers.
### High performance tools
In my previous company, Surfront. I developed a product which process million packets and parse HTTP request and DNS request, analyze and statistics domain and file data. Inject HTTP & DNS response packet. At first I use Java to implement it, performance is good, but it’s not enough for extremely case. So I use Go to reimplement it. Performance is extremely excellent.
### Support
I works closely with the customer. so I know what the customer wants.
I get to dig into all the logs and metrics to explain what happened, why it happened, why it wasn't detected sooner, and how you're going to make sure it never happens again. I finally writes a document to show the details.
Every time when I went out or on vacation, I need take my laptop with me. Even more, I will have data for internet access. I remember that there is a time, I take a tour to a famous site, I was on the bus, there is an emerge call, as I answered with phone call, I had taken out my laptop, to solve the problem.
### Versatile
I have a mix of different skills, like a webdev, software engineer, project management, support, data analysis, etc.
### Problem Soving
The copied file are randomly corrupted, it seemed that some content were mixed. I spent a few hours reading very carefully through all our related code. And realized that there is a utility method which was responsible copy stream data using buffer, and buffer variable is defined as java static variable. So When this method is called concurrently, buffer will be mixed up. Define the variable in local scope solved the problem. This gave me a big lession, I must carelly wrote code.
## Ownership
Tell me about a time when you took on something significant outside your area of responsibility
### Start-Up Company
My friend and I, we both are software engineers. we founded a company, surfront, which provides internet content security products for enterprise.
At first, just we two, we have no funds, no other resources. we developed all parts of our product from the ground up, all by ourselves. I was faced with a huge variety of tasks and constantly have to learn new things .
I might be writing database queries one day, architecturing the framework of application, designing a UI the next day, tuning the performance, do technical support, answering customer service emails the day after that.
But it was a great learning experience. I definitely learned a lot.
### Monitoring System
I'd like to talk about my work flow in my previous company, surfront.
I lead the project development. We had deployed totally thousands of servers for many customers in different locations. customers had responsibility to maintain & monitor the system, The servers are sometimes down due to network, hardware problems.
At first, We suggested our customers use a third-party tool to do monitoring. But customers may not have the resources to deploy it and maintain it. Such problems often bothered our customers and our support team.
When I heard the problem. I think I could build a system to help.
I researched server monitoring applications. including open source and commercial applications.
I developed a good solution to solve this problem. I integrated an open source server monitoring tool, called collectd. It will gather CPU/Memory/Disk/DiskIO/Traffic/Application metrics and send all metrics data to our global centralized monitoring system. I developed the centralized monitoring system to store all the metrics, and also providing fantastic dashboard to show metrics, server health information. The system also detected some problems in case we could notify our support engineers.
This solution was very helpful for both our customers and our support engineers, even for developers. and the monitoring system save significant costs, especially over the long-term. it also satisfied our customers to promptly discover problems and solve the problems, reduce the bundle of their responsibility .
## Earn Trust & Ownership
Give me an example of a tough or critical piece of feedback you received.
### Anti-Spam Database
I had a team member who was responsible for maintaining an anti-spam rules database. I was her supervisor. Once a time, she updated anti-spam rules database without doing a test. The spam database updated to customers, some customers complained that there were some false positives which some normal mails were classified as spam mails..
I think it's my fault. so I took responsibility to solve the problem.
At first, I immediately rolled back the updated database. I told the team member that she must follow the processes.
Then, I had to apologize to the customers, and promised to improve our processes.
Finally, The most important thing I thought was about fixing processes. Mechanisms for the long term rather than being focused on the short term incident. I built automation processes when spam database changed. without manually execution, automatically execute the test before updating the database.
The result is that such incidents never happen since then. The productivity was also improved. our customer was very happy.
## Deliver Results
### WAP Filter
I remember it was May 2010, China Telecom invited our company to develop WAP security project. The evaluation test will be held in two weeks.
At that time, Our company hadn’t had any product developed for WAP security. My partner and I had to decide whether or not developed the project.
After we effectively spent a few hours evaluating test specifications, we made fast decision to make it into a challenge.
In just two weeks, My partner and I, we two together developed the application from the ground up. I was responsible for developing content filtering and web UI. the development was very stressful. With two weeks of hard working, Most functions of the application work, but we hadn’t have time to do full testing.
The test lasted one week. During the test, My partner and I took different responsibilities. One check over the test items with the tester, one developing some undone requirements. After work hours, we kept developing in the hotel.
There are some functions we haven't implemented yet, for example, we need to scan MMS messages, but we didn't have the parser for MMS messages. Because there is no time to develop it by ourselves. so I asked my friend for help. My friend gave me a library so I can quickly implement the functions.
Finally, the test went well, we got first rate. We got orders from 6 provinces of China Telecom. After this project, we also got an opportunity to develop an important project to filter all of MMS messages for China Mobile.
## Dive Deep
### Java GC
In my previous company, I worked on an important project, which was deployed in China Mobile to filter all of MMS messages in China Mobile. The requirements are very strict. our application needs to handle 15,000 messages per second, and latency could not exceed 3 seconds.
I implemented MMS message filtering with high performance, robust and fault tolerant. Due to lack of resources, I didn't test the performance in extreme cases.
When the project was deployed in a production environment, In normal cases, the project works pretty well.
before rolling to product. The customer wanted to do all extreme cases to test whether the system could handle the largest throughput. I encountered a problem which is really weird. The system works well at the beginning. but randomly freezing for 2-20 seconds.
I had to investigate the problem and solve the problem.
At first, I tried to find the problem are caused by our implementation. I added metrics in all parts of message filtering. Such as how much time in processing, how much time in waiting. Gather such metrics data to analyze.
Then, I tried to use java profiler tool like JProfiler to find the bottleneck. Improve my code with more memory efficiency and fast execution.
At last, I realized that the problem maybe caused by Java GC. Because Java GC would pause the whole JVM when doing full GC. I added some options to enable gc logging in JVM. I analyzed GC logs, try to figure out the relationship between GC pause and our application processing throughput. Yes. I find the cause. I tuned every possible option and test, record data, compare results, find out the best option with the most performance.
The result also surprised me, because I thought that if you give your application more memory, the more performance your application will be, We gave 32GB to our application. But in this case, The problem is we define too much memory for our JVM, We just use 4GB. And choose a well performance GC strategy was also the key to solve the problem.
### Anti-Spam
in 2002, I joined a startup company, MessageSoft, developed a product for email security.
we quickly found out there are tremendous requirements to stop spam in the email security market. so we decided to develop anti-spam module into our product.
I was responsible for to design, implement and maintain the module. At the beginning, I had no idea how to do it.
So I began to study an open source application which filter spam messages, the application is called SpamAssassin. it uses Perl to implement, heavily use Regular Expression.
I learned a lot from this. so I came up with my own solution.
At first, I implemented a standard mail filtering language, which is called Sieve. it is a filtering language designed for mail filtering. so most of the filtering for to design, implement and maintain the module. At the beginning, I had no idea how to d rules could be expressed using Sieve.
My implementation of Sieve filtering language is flexible, extensible, high performance, it is easy to write rules in Sieve.
I started to analyze the data by isolating spam messages, once I had a good sample of these messages, I used statistics techniques to identify different patterns to classify these messages.
Because no one knows how to define the rules to stop spam. I also need to come up the base rules and develop tools to maintain ant-spam rule databases.
I analyzed a lot of spam messages. Conclude many rules into spam database. Which could filter about 95% spam messages.
I also designed the workflow to maintain the ant-spam rules database. Which could be easily done by a non-technical engineer.
## Invent and Simplify
### Automation
In my previous company, we used to spend a lot of time on manual build/release, manual testing, manually deployment. it's very costly for our limited resources.
I headed up this problem, implemented the whole automation process for our products.
I implemented build script using Ant. build different kinds of packages for development, test, production environment. such as build update packages, release packages, even build ISO for product distribution.
I integrated CI tool Jenkins, once a committed change. automatically build the project, run unit test and integration test.
then deployment was also automated. I implemented some roles in Ansible, which auto automatically setup and configure a brand new server to deploy our product, or update production environment.
As part of every deployment they go through several different kinds of testing. Started with integration testing. Browser and web based testing. Load testing.
Also I help setup monitor system to measure important metrics. Based on already-collected metrics, set alerts to points where certain thresholds are crossed and notify team members.
The result is we save a lot of time without waiting for a new build, preventing expected mistakes without manually update production environment. our customers were satisfied with our product.
### Restful API
Most of the applications I have developed are server side applications, which provide web UIs to allow user to access and manage our application. we use Java on the server side to generate web pages for users. This architect bundled the responsibilities for our Java developers.
To increase development productivity, I wanted to find a solution to separate frontend and backend, and define clear layers in both frontend and backend.
For backend, I defined repository layer to access databases, service layer to provide service method, web service layer to provide restful API, each layer depend on top another layer. and cannot across to another.
So do frontend, I incorporated VueJS as our frontend framework, frontend consumes RESTful API.
The architect is easy to start for new developer.
For single responsibility developer, he could only be responsible for either frontend and backend.
### Ownership
### Support Team
In my previous company, Surfront. We have support team. Because engineers in support team encountered with the same series of problems again and again, and support team would passed most problems to development team.
Every time I solved the problems, I wrote detail documents about how I solve the problems.
Sometimes I also invite support engineer to watch the whole processes.
and I periodically trained support engineers.
I also created a FAQ that provided customers with answers to the most commonly asked questions.
As a result, for normal problems, support engineers could handle most of the problem. developer will only responsible for problems related issues. As a result, our customer satisfaction increased, we don't need as many support, developer involved in support decreased.
## Have backbone:
Tell me about a time when you disagreed with a manager or someone of a higher level?
### disagree with boss?
If I have different ideas with my manager or boss, I will try to collect data or make a prototype trying to convince him. If I have tried my best and he still disagree, then I would respect his decision and start committing for that.
Our product which help our customer to save traffic of their bandwidth, we have traffic report which could show the trend graph in different period. but in a long period, the traffic is a little lower than a short period, so my boss want show maximum traffic instead of average traffic. Though I don’t agree the idea. I tried out to find some solution. At first I tried maximum and 95th percentile data. So I understand customers want to see the maxiumium. I tried to show such an obvious position. My manager were satisfied with my solution.
In my previous company, Our product focused on internet content security. There was a customer who had a special requirement which was not related internet content security. I protested to develop for this requirement. I believed that we should focus on internet content security because of limited resources. My manager made a decision to develop the project. I jumped on board with the plan, we worked hard to build the project from ground up. The project went really well. We generalized the project to product. That product became our most successful product.
### Conflict
Xujiaolin Video Info
I was managing a project and we were on a very tight deadline because we had to release a new version to our partner. The engineer that was assigned to the most important module was very talented, but unfortunately missed a deadline that I assigned. He was very frustrated.
I acknowledged that the deadlines were tight and explained again the reasoning and the importance of having the project ready for the release.
He told me about all of his other competing projects and how overwhelmed he was. I asked him if there was any way that I could help him come up with a solution. I helped him to analyze some obstacle problems
As a result, the engineer was able to focus on the project and meet the deadlines.
We successfully completed the project in time and received numerous compliments from both our customers.
### Project leader
I am in a lead technical role on several projects. In this role, it’s my responsibility to make sure the project is moving forward efficiently and break down any technical barriers to achieving our goals.
Every day, we had a quick stand up meeting. everyone in the team will talk about three things:
What did you do yesterday?
What will you do today?
What blockers stand in your way?
I had make sure my team member works as I expected.
I periodically review code. Mentor team member.
### Priority & Todo List
I make my daily to-do list! At the beginning of each workday, I write out tasks to complete, and list them from highest to lowest priority. This helps with my workflow and keeps me on track with what needs to get done for the day.
### Responsibility
Developed Ant scripts to build, developed automation and deployment utilities using shell script
Worked as a Linux administrator, maintained system in production, performed application and system level monitoring.
Responsible for compliance with coding standards, source code control, version control.
### Mistakes
My job has multiple conflicting priorities where it can be difficult to know what is most important and urgent. I worked out an important/urgent scale for rating tasks so that it is clear what takes the highest priority. If something is both important and urgent, it gets the highest priority. Important but not urgent is next and urgent but not important is next, then not important and not urgent is last.
As a result, my overall productivity in the past year has gone up considerably.
### Deadline
First, I’d check every possible way that could possibly make me hit the date, like using after-work time, weekends, or holidays, asking for help or suggestions, finding alternatives or simpler solutions, etc. Since I am the owner of the tasks, it’s my responsibility to deliver results on time. It’s my fault that I underestimated the workload when I took the tasks.
There was one time that we couldn’t finish the troubleshooting of a bug on time, the system should have sent out auto emails at 7:00 am but sometimes it failed. Before solving the problem, I will check each morning and manually trigger emails to the customer.
If I still couldn’t hit the date, I would look at features, pick out a few with highest priority that I could finish, and then discuss the problem with the project manager. Of course, I would apologize first and then try to discuss the best solution to minimize the impacts. Once the project manager agrees, I would notify all the people that could be impacted by the delay.
## Others
### Cache Product Development
In my previous company, I am co-founder, we are startup company.
Help customer to Cost Saving on expensive Bandwidth
Localizes popular Internet contents with high speed download experience
### Hardest Decision
The hardest decision I have ever had to make.
Great question, I think it's the time that whether I should join and co-found a startup company with my friend. At that time, I worked in a company. The position is stable and high paid. I also like the job.
I had to decide whether or not took the opportunity. the opportunity was risky, and maybe encounter many unknown problems. I make a hard decision to co-found the company. The company didn't have funding, I worked hard, and had done great contribution. The first few years, it is very hard, we developed a lot of projects and products to support the company.
I spent 12 years in the company. I also think this is my most wise decision.
### Risk
I had such experience which caused system downtime for a customer.
Because data in database were totally corrupted. so we had to rebuilt data from distributed servers.
From this experience, I learned that our system should be able to prevent the single failure of the system. add backup or easily recover solution.
### Find a Mentor and Being a Mentor
I have a partner which we have worked together for almost 20 years.
There is chinese famous quote "Give a man a fish, Feed him for a day; Teach a man to fish, feed him for a lifetime."
When I learn something, I will ask myself two useful questions to drill down into it: How does it work? and Why does this happen?
Mentored team members on principles, patterns, and technologies used in the application.
When I learn something, I also try find a mentor which could help me jump into fast lane. and also creates incentive for me.
For me, I am happy to share my knowledge