I was interested in learning how to run the dashboard after installing Devstack. The reason being that I need a method to check that Devstack installed correctly and the Devstack manual http://devstack.org/guides/single-vm.html mentions that you should be able to access the dashboard after installation.
jpich helped me figure this out. She's a core developer for Horizon and therefore knows a lot about the dashboard.
She told me to make sure port 80 is open on the firewall. She's not familiar with Vagrant, so we had to do some troubleshooting. She thought that since I could SSH into Vagrant, the port should already be open.
-
She thought there might be a configuration issue with the Ubuntu firewall and suggested using the Ubuntu tool ufw to configure it.
https://help.ubuntu.com/community/UFW
She suggested trying
sudo ufw allow 80to open up port 80 andsudo ufw statusto view the current configuration.I tried to use
ufwas she suggested: https://gist.github.com/terriyu/6201847 -
She suggested I try running
ifconfiginside the Vagrant VM to check my network configuration. The results of running the command are here: https://gist.github.com/terriyu/6198158Based on this output, she suggested that I try to point my external web browser (on my host machine not the VM) to
http://10.0.2.15but I got a "webpage is not available" error, even after usingufw. -
Next, jpich suggested running
curl http://localhost/on my VM and seeing what the output is. I did see a dashboard in the output: https://gist.github.com/terriyu/6201870So it appears that the dashboard is up and running locally on the VM.
-
jpich then realized that the problem is Vagrant has its own way of doing networking: http://docs.vagrantup.com/v2/networking/forwarded_ports.html
I tried to quit my VM to change my Vagrantfile to incorporate the port forwarding configuration shown in the above documentation. But the Vagrant VM hung for a long time when I tried
vagrant halt. I tried to shut it down using VirtualBox, but that didn't work. Eventually, I gave up and destroyed the VM. I'll have to make a new one.
Previously, eglynn explained to me a little about how the Ceilometer API works. An HTTP GET request is made, for example
GET /v2/meters/cpu_util/statistics?q.op=gt&q.op=lt&q.value=2013-07-18T15:32:26&q.value=2013-07-20T20:35:36&q.field=timestamp&q.field=timestamp
URL fragments like &q.op=lt are query parameters. Then the URL in the GET
request is decoded by WSME.
WSME stands for "Web Services Made Easy" and it's a Python package that makes writing REST APIs easier: http://pythonhosted.org/WSME/
jd and I decided to have regular weekly meetings. This is the first one.
We talked a lot about how the API works. I was looking in detail at the Ceilometer documentation to see how API queries work: http://docs.openstack.org/developer/ceilometer/webapi/v2.html#api-and-cli-query-examples
There are two ways to query the Ceilometer API. You can either
- Pass the parameters of the query in a URL to an HTTP GET request
- Pass the parameters of the query in JSON encoded data to an HTTP GET request
One of the most popular, easiest way to communicate via HTTP protocol is to use the command line tool cURL: http://curl.haxx.se/
With cURL, you can do both methods (1) and (2). The Ceilometer documentation has examples for both methods using cURL.
jd showed me an example for using cURL to send a JSON encoded query.
curl -H 'X-Auth-Token: <inserttokenhere>' -H 'Content-Type: application/json' -d '{"q":[{"field": "timestamp","op": "ge","value":"2013-04-01T13:34:17"}]}' http://localhost:8777/v2/meters
He said that the -d flag indicates data to be sent.
A URL-based query might look like:
curl -H 'X-Auth-Token: <inserttokenhere>' "http://localhost:8777/v2/meters/instance" "?q.field=metadata.event_type&q.value=compute.instance.exists" "&q.field=timestamp&q.op=gt&q.value=2013-07-03T13:34:17"
The symbols ? and & tell the application at the other end how to interpret
and parse the request.
You can also use the Ceilometer CLI (command line interface) but I don't see any examples of how to do querying with it. You can get a little documentation by typing
$ ceilometer --help
jd explained that you can use the command line tool telnet to "talk HTTP."
telnet is not secure like SSH, but it is still a very useful tool. It allows
you to connect to a TCP port, and HTTP is based on TCP.
jd showed me how to do a few simple things with telnet. For example, you can do
$ telnet openstack.org 80
Trying 174.143.194.225...
Connected to openstack.org.
Escape character is '^]'.
Connection closed by foreign host.
The connection is closed after about 15 seconds, if you don't communicate.
You can send a request over telnet. Trying connecting to the server again.
Once connected, type the request GET /foobar HTTP/1.1 and press ENTER twice.
$ telnet openstack.org 80
Trying 174.143.194.225...
Connected to openstack.org.
Escape character is '^]'.
GET /foobar HTTP/1.1
HTTP/1.1 404 Not Found
Server: Apache/2.2
Content-Type: text/html; charset=iso-8859-1
Date: Sat, 10 Aug 2013 21:43:58 GMT
Connection: Keep-Alive
Set-Cookie: X-Mapping-bffmijpk=5239B52D865E75C5414CC386D0915C5E; path=/
Content-Length: 273
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL /foobar was not found on this server.</p>
<hr>
<address>Apache/2.2 Server at 174.143.194.225 Port 80</address>
</body></html>
When you ask for a page that doesn't exist, you get the famous 404 error.
I asked jd what Keystone is for, since I notice that it's referenced in the Ceilometer API queries. He said:
Keystone is for authentication
it provides an API over HTTP too, like Ceilometer
you give keystone a username and a password
and it gives you a token
with that token you can prove you are terriyu to Ceilometer
this is our authentication mechanism
Keystone is called the identity server in OpenStack
so Ceilometer just interact with it to check you token
and that's it
I noticed in the Ceilometer API documentation that there were examples of the data being in both JSON and XML. jd explained:
we use WSME as a framework to build the API
and WSME understand XML and JSON
everybody prefers JSON nowadays, so you'll rarely encounter XML
I asked jd why people prefer JSON over XML and he said:
for one, XML is a bit complicated and over engineered
and JSON is understood natively by all browsers
JSON stands for JavaScript Object Notation if I'm right
so in a web page, you can have JavaScript code requesting the ceilometer-api using JSON
and getting JSON from it
in a snap
doing this with XML is possible, but much more painful, much more work to do, because XML is not a native type in JS
you would have to parse it, then split it into javascript objects…
I've had the problem where I try to shut down the Vagrant VM gracefully without
losing my state using vagrant halt, but the command hangs. Then I try
various things to stop the command like keyboard interrupt, killing the process,
going into VirtualBox and stopping the process from there, but eventually I
end up destroying the machine and losing my state.
jd suggested that I might try just using VirtualBox and avoiding Vagrant.
jd says that most of the time you don't need Devstack to test Ceilometer. In
fact, he frequently tests Ceilometer by just running tox on the Ceilometer
Git repository. He only needs Devstack if other components of OpenStack are
involved.
I told jd that I had problems when I installed 10gen's version of MongoDB and
then ran ./stack.sh He said that I should avoid using vendor packages on
Debian/Ubuntu. For this case, it's an issue of bad design.
typically in this case, they used a different name than the package provided by Ubuntu and Debian
but they provide the same set of files
and that is NOT possible and NOT a good thing to do
2 different packages can't provide the same file(s)
He said another reason to avoid vendor packages is because they are usually not as well tested and miss dependencies
The blueprint is here: https://blueprints.launchpad.net/ceilometer/+spec/api-group-by
In the Ceilometer docs, I noticed that you can aggregate over a period of time. In this section http://docs.openstack.org/developer/ceilometer/webapi/v2.html#functional-examples, I see
You may want to aggregate samples over a given period (10 minutes for example) in order to get an array of the statistics computed on smaller durations:
GET /v2/meters/cpu_util/statistics
q: [{"field": "timestamp",
"op": "ge",
"value": "2013-06-01T00:00:00"},
{"field": "timestamp",
"op": "lt",
"value": "2013-07-01T00:00:00"},
{"field": "resource_id",
"op": "eq",
"value": "64da755c-9120-4236-bee1-54acafe24980"}]
period: 600
I mentioned to jd that this sounds like a "group by" and he said that's exactly right.
it groups by timestamp
for group that are N seconds long
the different is time are ranges, and the GROUP BY you'll have to implement is not about range, but about simpler values
*difference
but the spirit is the same, I agree
For the group by blueprint, we want to be able to group on any number of fields, though being able to do it on a single field is a starting point.
Another thing I was trying to figure out is how the storage drivers are called by the API. I saw that a hook is used to get the storage connection:
But that's not where the storage drivers are called.
They're called in ceilometer/api/controllers/v2.py. I see there is an
import statement to get the storage drivers
from ceilometer import storage
Reference: https://github.com/openstack/ceilometer/blob/master/ceilometer/api/controllers/v2.py#L45
Then there are calls to the methods in the storage drivers. For example:
@wsme_pecan.wsexpose([Sample], [Query], int)
def get_all(self, q=[], limit=None):
"""Return samples for the meter.
:param q: Filter rules for the data to be returned.
:param limit: Maximum number of samples to return.
"""
if limit and limit < 0:
raise ValueError("Limit must be positive")
kwargs = _query_to_kwargs(q, storage.SampleFilter.__init__)
kwargs['meter'] = self._id
f = storage.SampleFilter(**kwargs)
return [Sample.from_db_model(e)
for e in pecan.request.storage_conn.get_samples(f, limit=limit)
]
Reference: https://github.com/openstack/ceilometer/blob/master/ceilometer/api/controllers/v2.py#L462
You can see the call to pecan.request.storage_conn.get_samples(f, limit=limit)
For the blueprint, we are more interested in the get_meter_statistics()
method.
@wsme_pecan.wsexpose([Statistics], [Query], int)
def statistics(self, q=[], period=None):
"""Computes the statistics of the samples in the time range given.
:param q: Filter rules for the data to be returned.
:param period: Returned result will be an array of statistics for a
period long of that number of seconds.
"""
kwargs = _query_to_kwargs(q, storage.SampleFilter.__init__)
kwargs['meter'] = self._id
f = storage.SampleFilter(**kwargs)
computed = pecan.request.storage_conn.get_meter_statistics(f, period)
LOG.debug('computed value coming from %r', pecan.request.storage_conn)
# Find the original timestamp in the query to use for clamping
# the duration returned in the statistics.
start = end = None
for i in q:
if i.field == 'timestamp' and i.op in ('lt', 'le'):
end = timeutils.parse_isotime(i.value).replace(tzinfo=None)
elif i.field == 'timestamp' and i.op in ('gt', 'ge'):
start = timeutils.parse_isotime(i.value).replace(tzinfo=None)
return [Statistics(start_timestamp=start,
end_timestamp=end,
**c.as_dict())
for c in computed]
Reference: https://github.com/openstack/ceilometer/blob/master/ceilometer/api/controllers/v2.py#L556
jd told me to follow the tenants of test-driven development and write the tests first.