Ive created a js app that can be used to test solutions for the community. Certain ones people made for challenge #1 wont be able to be put through it since they accept user input through entering it while its running rather than command line arguments but ill be enforcing command line argument use going forward

Let me know if theres any issues you find with it. I tested with with javascript, python, and rust and seemed to be working great. There can be differences in runtime when you run the same thing multiple times so ill be taking the median of running it 50 times for the performance score in the challenge

  • Andy@programming.dev
    link
    fedilink
    arrow-up
    1
    ·
    1 year ago

    So is the idea to run our own solutions through it locally and post the results alongside our submissions?

    Does it permit providing all the test cases as a long argument list to do it all in a single invocation, or is that cheating?

    I find hyperfine very nice for casual benchmarks, FWIW.

    • Ategon@programming.devOPM
      link
      fedilink
      English
      arrow-up
      1
      ·
      edit-2
      1 year ago

      People can use it to test their solution locally but for final results it will be ran by the people running the challenge (since test cases are hidden + so people can’t fake a time)

      You can enter as many test cases as you want into the txt file and it’ll run through them all. You need to make your own though

      • Andy@programming.dev
        link
        fedilink
        arrow-up
        1
        ·
        1 year ago

        Thanks!

        Regarding my second question, I’m thinking for some languages there’s a performance hit each time the program/function/runtime is launched, and that it’s more performant to handle all tests with a single invocation that loops through them all.

        But I think, and am looking to confirm, that that form is not allowed for these challenges, so we should only have each solution take a single test case.

        • Ategon@programming.devOPM
          link
          fedilink
          English
          arrow-up
          1
          ·
          1 year ago

          I have it coded where it will check how long it takes for a basic program to launch and subtract that from the time

          But yeah solutions should only take 1 test case at a time