|
Preparing for a Hackathon
This section outlines recommendations gathered from participants and mentors over the years on how to prepare for a hackathon. While each team will proceed differently in
practice, we find that teams who follow these suggestions typically have the greatest success.
Team Introductions:
In the weeks leading up to the event, the hackathon organizer will send an email introducing your team to the mentor(s) with whom you will be working. The organizer of
the hackathon will schedule a introductory web meeting with the team and mentor(s) to address the following topics:
- Team/Mentor introductions
- Discuss code(s) your team will be working on (give code access where appropriate)
- Determine which compute system(s) your team will use during the event
- Prepare and profile your code for the event
- Assign any action items based on discussions
This initial meeting will help get your team organized and ensure that everyone is on the same page for follow-up discussions.
Preparing Virtual Tools
Virtual Hackathons will take place online using Zoom. You will need to install their client, which is available for Windows, MacOS and Linux, and make sure that it is updated to the latest
version. Once you have installed the Zoom client, you can test your microphone and camera interface with Zoom here. You can find more information on Zoom system requirements, including bandwidth requirements, here.
Zoom: Download and get started
We will also use:
Slack: Download and get started. Slack will be used as the main place for announcements to all participants, where to ask for help, as well as the main group chat (e.g., chat, code sharing)
Zoom chat is disabled. Make sure to turn on your notifications. A Slack channel will be created for each team.
Gsight: Download
Google Drive: link to be sent pre-event, to upload team presentations
Obtain Access to the Hackathon Compute System
Three weeks prior to the event each team is provided with access to a GPU system to be used at the event. This gives you the opportunity to familiarize yourself with aspects of the system that might be new to you (e.g. different batch scheduler, job launcher, etc.) and also to get your application compiled and running on the system. This helps to ensure your team is ready to start programming GPUs on the system when you arrive for the event, instead of spending time learning how to use the system.
Preparing Your Code to Run on the Hackathon Compute System
Although you might be using your application for production work already, that does not ensure it is ready for the work you will be performing at the hackathon. For example, if your code takes four hours to produce results, it will be difficult to test the many incremental changes you will likely be making to your code during the event. To address this point, this section outlines recommendations for preparing your application for the event.
Kernel or Mini-Application
Ideally, your application should be limited to a few thousand lines of code. If you are working with a much larger application, it is preferable to extract specific kernels or a “mini-app” that contains only the relevant parts of the full application whenever possible. This makes the code more manageable for all your team members (including the mentors - who are likely not familiar with the code) and helps to eliminate potential problems with other parts of the code that are unrelated to the work being performed. Once you have your reduced application running on the GPUs, you can add the changes you made into the full application to understand how it speeds up (or slows down) the application as a whole.
Self-contained Code
Your code should also be self-contained whenever possible. By eliminating external dependencies (e.g. netCDF), you will not need to rely on specific packages (or specific versions of packages) being available on the hackathon system. To do so, you can include any code needed for external dependencies within (or alongside) your application. If this is not possible/practical, you should make the dependency known to the local hackathon organizer ahead of the event so it can be installed. In addition, your build system should be free of any specific system dependencies (e.g. Cray computing environment). Removing these external and system-specific dependencies will make it easier to get your code running on the hackathon system.
It is also important to understand any dependencies that might arise due to your choice of programming model. For example, if you plan to use OpenACC to target GPUs, you will likely want to use the PGI compilers or GCC, so you need to make sure your code compiles with PGI. Doing so before the event helps to ensure your time at the hackathon is actually spent on GPU work (not getting your code compiled). Your mentors should be able to help you identify these types of dependencies.
Compile, Run and Profile on a Hackathon GPU System
After making any such changes to your code (e.g., using different compilers, extracting kernels, building your own libraries), you should always confirm that your application still compiles, runs, and (of course) gives the correct results. Ideally, this should be done on the system(s) you will be using during the event. This workflow (make changes - compile - run - check results) will be used frequently during the week of the event, so it is important that the process is efficient. In general, you should configure your application to run in ~30s, and only run it on a single process (if possible).
Simplifying Your Workflow: Tree diagram, Profile, Correct Results
As mentioned above, it is likely that the mentors working with your team are not familiar with your application. So, it is helpful to have a way of describing your application’s program flow to them (e.g. call tree, flow chart, etc.). Helping your mentors (and all team members) understand your code structure before the event can make them more efficient and save you a lot of time during the event.
Having a profile of your application, which shows details about how much time is being spent in different regions of the code, allows you to identify the most beneficial regions to accelerate. For example, you want to ensure you are spending your time accelerating parts of your code that account for a sizable percentage of the total runtime; optimizing regions that only account for 2% of the total runtime will not gain you much. If you need help, your mentors can likely point you in the right direction on generating a profile. Having this available before the event is helpful in planning for the week.
Another important aspect of preparation that is often overlooked is having a way to verify correctness of your results. This is an important part of the hackathon workflow, so arriving with an automatic way of doing so (e.g. a correct results file which can be compared against new results with diff) can save more time for development. It is not uncommon for a team to get their code optimized and running blazing fast on the GPUs only to find that it is not giving the correct results!
| 