docs: add requirement analysis #1
No reviewers
tobias.kantusch
felix.auringer
jann.stute
toni.poredda
Labels
No labels
Compat/Breaking
Kind/Bug
Kind/Documentation
Kind/Enhancement
Kind/Feature
Kind/Security
Kind/Testing
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Reviewed
Confirmed
Reviewed
Duplicate
Reviewed
Invalid
Reviewed
Won't Fix
Status
Abandoned
Status
Blocked
Status
Need More Info
No milestone
No project
No assignees
5 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
kraftwerk-ctf/kraftwerk!1
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "docs/requirements"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
docs: add requirement analysisto draft: docs: add requirement analysisdraft: docs: add requirement analysisto WIP: draft: docs: add requirement analysisWIP: draft: docs: add requirement analysisto WIP: docs: add requirement analysis@felix.auringer This is not yet complete but it would be great to have a disucssion on what we want to achieve (if we want to achieve that). 😊
@ -0,0 +12,4 @@The CTF platform should be designed so that a multitude of requirements regarding challenges and usage is considered.When designing the platform, we for now only consider jeopardy-style CTFs (no attack-defense-style CTFs)### Users and AuthenticationI would add groups here. We should support at least player and administrator as different groups and all authentication possibilities should provide these.
I added a paragraph on roles.
@ -0,0 +14,4 @@### Users and AuthenticationUsers must be able to log in to the platform via any OIDC-compatible IdP.I think there should be an interface with an implementation for OIDC. Forcing OIDC would maybe prevent smaller CTFs from using it because they do not want to set up any IdP.
This is up for discussion. Added a paragraph on a authentication method that works without OIDC.
@ -0,0 +15,4 @@### Users and AuthenticationUsers must be able to log in to the platform via any OIDC-compatible IdP.Users must be able to create and join teams with an invite-link or token.Users must be able to create teams and join them with an invite-link or token.
@ -0,0 +19,4 @@The platform must support both competitions for teams and individuals.The entity that is credited for the scores is called an Account.For integration with other services (like Discord bots and support), the user and account must have secret tokens that can be used to prove access to the user and account.Should changing the username be a part of the API or the authentication before that? (I cannot comment on every line, so this comment is on the line above 🤷♂️)
Removed the requirement.
@ -0,0 +17,4 @@Users must be able to log in to the platform via any OIDC-compatible IdP.Users must be able to create and join teams with an invite-link or token.The platform must support both competitions for teams and individuals.The entity that is credited for the scores is called an Account.Is this (especially naming) really part of the requirements?
For clear naming within the requirements document: yes.
@ -0,0 +27,4 @@Challenges are the main part of the platform. Challenges consist of one or more flags.Flags must be customizable (=flag instancing) to the account so that cheating can be detected more easily. But hard-coded flags must also be supported.#### PrerequisitesNot sure whether this fits here but I would also like to have challenges that unlock depending on other conditions, i.e., at a specific time.
Good idea, added a sentence on other prerequisite types.
@ -0,0 +29,4 @@#### PrerequisitesIt must be possible to allow accessing challenges only after a set of other challenges were completed. It must not be possible to gain information about challenges that are not unlocked.Should we also differentiate between completely hidden and locked-but-announced challenges?
Good idea, we had this discussion with the old platform as well.
@ -0,0 +60,4 @@* Challenge IP and Ports as well as a hint to what this kind of service is (maybe a short description)Exposing information like ports and IP must be possible for all types of interactive challenges.### NetworkingI would add raw port exposal. Definitely with a big warning in the docs but I think it further lowers the bar for adoption.
I think raw port exposal is not something, we should support, as it can be too easily enumerated from other users and, thus, causes more trouble. Furthermore, there are enough settings (e.g. in a university) where not all ports make be exposed.
@ -0,0 +62,4 @@### NetworkingAs challenges of also require fiddling with the network, the platform must support a variety of different network operations starting at ISO / OSI layer 2 to allow for challenges that, e.g., require ARP spoofing.often?
@ -0,0 +93,4 @@The exact interface has yet to be defined as it must be common functionality subset across orchestrators (or provide an extension point).Examples for orchestrators are a Kubernetes orchestrator or a Docker orchestrator.### User-EnvironmentI like the Flugzeug but I would give it a very low priority 😅
Priority is not somehting modeled in this document, yet.
@ -0,0 +77,4 @@#### VPN AccessSome challenges cannot be solved without having access to the internal network.Therefore, access to the isolated challenge network "playground" is necessary were users can interact with challenges as they wish.Is the playground network an account-based network or a challenge-instance-based network?
I removed the
"playground"part. I meant the whole network that is reachable for the account.@ -0,0 +105,4 @@* TODO: Updating the base image### ScoringMaybe add that it should be possible to select the scoring algorithm? On the congress, there was some feedback that our old concept was gameable. There also might be CTFs with a fixed number of participants where a scoring algorithm using that additional information could be benefitial.
Added. However, we need to discuss details on whether how exactly the algorithms work and whether we need to support algorithms per challenge.
@ -0,0 +124,4 @@#### NotificationsThe administrators should be able to post global notifications and notifications regarding specific challenges.The user must be able to acknowledge these notifications.Why is acknowledging those important? 😅
Clarified.
@ -0,0 +139,4 @@## Non-FunctionalApart from functional requirements, we collected a set of non-functional requirements that make working with the platform (TODO: find word).pleasant?
@ -0,0 +11,4 @@## ImplementationThe implementation can happen with [OVN](https://www.ovn.org/en/).There, we can have a central database (with ``northd`` and `southd`).What's with the double backtick here?
Removed.
@ -0,0 +37,4 @@L2TP/IPsec provides another well-supported VPN solution that can be configured from the major operating systems (even with GUI!).For the built-in VPN solution, we could provide config files (profiles on macOS or a config for importing with `nmtui`) for simple configuration.The issue is that L2TP, despite it's name, only tunnels [PPP](https://en.wikipedia.org/wiki/Point-to-Point_Protocol).It seems hard to tunnel all Ethernet frames via the interface.We do not want all ethernet frames, right? Only the ones for which an upper layer has chosen the vpn interface based on routes or for which the user has explicitly chosen the interface.
Clarified.
@ -0,0 +42,4 @@### L2TPv3A solution to the layer 2 problem would be [L2TPv3](https://datatracker.ietf.org/doc/html/rfc3931).However, being standardized in 2005, it still has no support in major operating systems but in some of the enterprise swtiches only.switches
cfc6bfd928tofb1e6d190fWIP: docs: add requirement analysisto docs: add requirement analysis@ -0,0 +125,4 @@Starting challenges must be an automated process in case of interactive isolated challenges.Challenges can consist of multiple services that interact with one another.As CTFs and the number of interactive challenges differs in size, there should be multiple different orchestration services that take the responsibility to start and stop challenges in virtual environments.I think that's not grammatically correct, I would suggest
Done.
@ -0,0 +152,4 @@The platform must support score groups.Score groups are a mechanism for multiple, completely independent groups to participate in a competition without interfering with each other.Examples are age groups where only specific age groups are eligible for winning prices.Scoreboards should be visible for all score groups.I assume this is referring to all scoreboards being visible to all score groups?
Maybe change to:
I would word it to:
To make it configurable
I think configuring which score group may see which scoreboards might be an overkill.
I now defined four access levels for every scoreboard.
Yes, I think this is a nice way to implement it
@ -0,0 +59,4 @@### Custom commandsIt could also be possible to provide commands that tunnel a user to a jump host that is provided by the platform.One disadvantage of this approach is that it only work for a limited number of OSs and can cause confusion.typo:
Fixed.
@ -0,0 +68,4 @@The platform must support the following (not exhaustive) list of challenge types:* Interactive Challenges* Isolated Challenges: Multiple independent instances of challenges can be started, once per account (i.e., team or user). These have their own state.The first point is already covered: Random subdomains must be reproducible per account and the internal challenge IPs are the same of all accounts. I added an additional notice that the latter must apply also after a restart.
Regarding the second point: There is currently no requirement that challenges must be stopped after solving them. Is that an additional requirement?
Stopping challenges, or at least handling handing in the last flag, should be a requirement.
@ -0,0 +147,4 @@* TODO: Updating the base image### Scoringnot sure where else to put this, so here it goes
Regarding the first point: I added a requirement that CTFs are split into three time periods where the last ensures access to the same information as during the playtime.
Regarding the second point: I added a requirement for that next to the scoreboards.
We discussed whether how the second point could be handled. We decided that this is a property of the score group (score gorup could have a property like
disclose_solveswhich would trigger showing the solves for all challenges).@ -0,0 +14,4 @@### Users and AuthenticationUsers must be able to log in to the platform via one of these (mutually exclusive) options:Maybe have this not exclusive for this, but rather keep it kind of open to allow for implementation of e.g. LTI or Flaghub
another example would be OAuth2.0 in case login with ctftime would be desired
I added
at leastto stress that the two listed mechanisms are not an exhaustive list.@timo.neyer What exactly do you mean with keeping it open? Would you like support for having multiple mechanisms at the same time? I would rather delegate such complicated behavior to an IdP like Keycloak.
I would just implement it with a bare minimum of abstraction so that it can be extended to whatever is needed. But It should, as you said, only be possible to have one login option per CTF. Everything else would result in too much work for too little gain
@ -0,0 +193,4 @@### SetupThe setup should be reasonably easy and straightforward. Having to debug multiple different applications and integrations to get started is a blocker for users.Very similar to Hosting and Deployment (via Nils).
I removed and/or rephrased duplicate content.
@ -0,0 +133,4 @@The exact interface has yet to be defined as it must be common functionality subset across orchestrators (or provide an extension point).Examples for orchestrators are a Kubernetes orchestrator or a Docker orchestrator.As we want to allow re-using infrastructure for orchestration (from our experience, it is quite painful to manage multiple Kubernetes instances for different settings), orchestrators must be able to orchestrate for multiple CTFs / APIs.Should it be possible to register multiple instances of one orchestrator type? E.g. spread the challenges across multiple kubernetes clusters or one cluster per CTF.
I decided that it doesn't hurt to have multiple instances of the same orchestrator (e.g., mutliple k8s clusters).
I am not sure whether it is a good idea to allow multiple orchestrators per CTF.
How would the API decide which orchestrator to choose for a challenge?
For now, I would leave this undefined and not explicitly allow multiple orchestrators, as we're not clear how different orchestrators can be. For instance, networking would have to be equal across all orchestrators.
@ -0,0 +180,4 @@Larger setups for multiple hundred to thousands of participants may need another setup routine.All components must be containerized, although containers may need access to host resources directly (such as the host's network).It should be possible for the API to support multiple CTFs.It should be possible to impose resource limits on CTFs (e.g. only very few instances for archived CTFs).
Added a sentence requiring resource limits.
We also added a sentence on the difference between team wide and CTF wide resource limits.
@ -0,0 +61,4 @@#### SteppingChallenges should be available in steps, i.e., if a challenge requires multiple steps to solve, the author should be able to “unlock” new descriptions after each step, depending on how much info the author wants to give.Isn't this basically multiple challenges that only unlock after solving the previous ones? I don't see how you'd detect that people got past a certain point to trigger the unlocking of new descriptions. A flag hand-in on the other hand would be a very easy trigger. Alternatively, you could have one challenge with multiple flags and unlock more descriptions after each flag hand-in. In any case, this requirement could be made a bit more specific as to what exactly the idea is here.
I had assumed something similar to the way liveness and readiness probes work, where the author would specify a command to test whether the checkpoint has been reached
I think what we had in mind is indeed releasing additional information after handing in a flag.
I added this as an example step to make it clearer. I'm not against other steps but I think flags are the only way to implement this without major overhead.
The pictured use case is educational: After solving part of the challenge, it would be possible to provide explanations of the previous step / hints for the next step.
If so, the challenge would need to have a tamper-proof way of sending events to the outside.
Changes the whole stepping thing to a "could".
@ -0,0 +58,4 @@### Custom commandsIt could also be possible to provide commands that tunnel a user to a jump host that is provided by the platform.Elaborating on the idea of a jump host: Why not have a jump host for users to connect to that is dual-homed, i.e. reachable from remote (either public or behind L3 VPN), as well as being connected to the challenge network? I believe this should be easier to implement than an L2 VPN.
Some points worth considering:
As far as I know, it is not easily possible to route ethernet frames through SSH. It would probably be possible to achieve it via tap devices but at that point, we could also use something like GRE.
Everything on Layer 3 can be achieved with wireguard, so a jump host is not really necessary, right?
@toni.poredda n.b.: SSH only supports TCP tunneling.
FYI, merging now but feel free to open issues for discussion.