Alright, let’s talk about how I wrestled with calling Doris from, well, pretty much anywhere. It was a journey, let me tell you.
First off, I needed to talk to Doris. Obvious, right? But the “how” was the tricky part. I started simple. I figured, “Hey, it’s just an HTTP endpoint, I’ll just use `curl`!” Wrong. Well, not entirely wrong, but definitely not the whole story.
So, I started crafting my `curl` command. Something like this:
curl -X POST -H "Content-Type: application/json" -d '{"sql":"select from my_table limit 10;"}' http://doris-host:8030/query Easy peasy, right? Nope. Got some weird errors back about authentication. Doris wasn’t just letting anyone poke around. Fair enough.
Next step: figured out I needed to add authentication headers. Dug around in the Doris docs (which, let’s be honest, aren’t always the clearest things in the world), and found out I needed to add a `Authorization` header with a “Basic” auth scheme.
So, tweaked my `curl` command:
curl -X POST -H "Content-Type: application/json" -H "Authorization: Basic $(echo -n 'user:password' base64)" -d '{"sql":"select from my_table limit 10;"}' http://doris-host:8030/queryOkay, getting warmer. This time, I got a different error. Something about needing to enable the HTTP protocol. Turns out, Doris has a few different ways to accept queries, and the HTTP one wasn’t enabled by default.
Had to dive into the Doris configuration files (*, to be exact) and add/modify the `http_port` setting. Restarted the Doris FE (Frontend) process, and BAM! Got a response back. Success! Sort of.
The response was a JSON blob, but it wasn’t exactly user-friendly. It was a nested structure with metadata and the actual data buried deep inside. I needed to parse that JSON to get to the good stuff. Enter `jq`! (If you don’t know `jq`, learn it. It’s your best friend for dealing with JSON from the command line.)
So, my final `curl` command, piped through `jq`, looked something like this:
curl -X POST -H "Content-Type: application/json" -H "Authorization: Basic $(echo -n 'user:password' base64)" -d '{"sql":"select from my_table limit 10;"}' http://doris-host:8030/query jq '.*'That got me the actual rows of data. Much better. But still not perfect. I wanted to format it nicely. Turns out `jq` can do that too! Played around with the formatting options until I got something that looked decent.
But wait, there’s more! I wanted to automate this. I wanted to put this `curl` command into a script, so I could easily query Doris from anywhere. And that’s where things got interesting.
Turns out, the `Authorization` header with the base64 encoded username and password wasn’t the most secure thing in the world. Anyone who could see the script could see the credentials. Not good. I needed a better way to handle authentication.
After some digging, I discovered that Doris supports using tokens for authentication. Much more secure! The process was a bit involved (generating a token, storing it securely, passing it in the `Authorization` header), but it was worth it for the peace of mind.
So, my final solution involved:
- A script that retrieves the Doris token from a secure location (like a password manager or a dedicated secrets store).
- A `curl` command that uses the token in the `Authorization` header.
- `jq` to parse and format the JSON response.
And that, my friends, is how I call Doris from anywhere. It wasn’t a walk in the park, but I learned a lot along the way. The key takeaways? Read the docs (even if they’re not perfect), don’t be afraid to experiment, and always prioritize security.
