Support / questions about Lemmy.
I would start by kubectl exec'ing into your pod and using nsllookup/dig/whatever you have available to check whether DNS resolution is working inside there at all. I don't think there is an easy way to break DNS resolution via config without it being pretty clear that you're doing so, but you could try messing with the pod spec's dnsPolicy (I often end up using ClusterFirst) and dnsConfig.
I may be able to help more if you post your pod spec.
Thanks. I shelled into my container and I was able to successfully do DNS requests (and full HTTP with the outside world).
0: error sending request for url (https://lemmy.ml/.well-known/webfinger?resource=acct:cryptography@lemmy.ml): error trying to connect: error:16000069:STORE routines:ossl_store_get0_loader_int:unregistered scheme:../crypto/store/store_register.c:237:scheme=file, error:80000002:system library:file_open:reason(2):../providers/implementations/storemgmt/file_store.c:267:calling stat(/usr/lib/ssl/certs), error:16000069:STORE routines:ossl_store_get0_loader_int:unregistered scheme
(and so on)
My podspec is as follows:
volumes:
- name: lemmy-config
secret:
secretName: lemmy-config
imagePullSecrets:
- name: ocirsecret
automountServiceAccountToken: false
containers:
- name: lemmy
image: [...]
imagePullPolicy: Always
env:
- name: LEMMY_CONFIG_LOCATION
value: /etc/lemmy-config/lemmy.hjson
ports:
- containerPort: 8536
resources: {}
securityContext:
capabilities:
drop:
- CAP_MKNOD
- CAP_NET_RAW
- CAP_AUDIT_WRITE
volumeMounts:
- mountPath: /etc/lemmy-config
name: lemmy-config
readOnly: true
- name: lemmy-ui
image: [...]
imagePullPolicy: Always
ports:
- containerPort: 1234
resources: {}
securityContext:
capabilities:
drop:
- CAP_MKNOD
- CAP_NET_RAW
- CAP_AUDIT_WRITE
enableServiceLinks: true
hostname: lemmy
restartPolicy: Always
Yeah, I think you were spot on in your diagnosis of that particular error in your updated OP of needing to update-ca-certificates. As far as I can tell based on that podspec you aren't really doing anything particularly odd that I would expect to break DNS or something at the network layer.
Is the issue you are seeing in your logs any different now, or still the same as before the ca certs?
Make sure you don’t have private instance checked. No idea what it does but I noticed in the logs you can’t have both private instance and federation checked. I’m guess private is just that, no federation, just a walled garden, possibly for non-prod and test.
Leave allowed instances blank. Only use blocked to defederate ones that are undesirable.
Start searching instances and communities in the search feature. This will reach out to them and start syncing posts and content.
Also, may want to look into 0.18.0. It doesn’t use web sockets for federation anymore. Which is a lot more efficient and I’m seeing less syncing issues with things like comments.
Also don’t know why numbers don’t show in my replied in mobile
Thanks. I'm not private and that is blank. All searches fail, although it might be federating in the background.
I haven't upgraded to 0.18.0 yet because the Dockerfile I wrote uses the crates.io release to build Lemmy from source but they haven't published 0.18.0 on crates.io yet. I can change that to git though. I'll probably do that tomorrow and see what happens.
How are you searching? The best way I found to load up my instance was to head to this site
Click the house in the top right and input your instance domain. This will reformat everything to the right search term.
Then click the little copy link (it’s the one under the primary instance name). And paste that into searc and mash search a few time. The instance will appear in search after a few tries (heavier trafficked instances are slower to respond). Then just open the instance and subscribe. That will start syncing the community. Your federated instances will slowly grow from there.
Yes, searching is what is not working.
Yes, I've tried searching with the correct format. I also tried directly navigating to a community from mine via a URL, which threw this interesting error:
LemmyError { message: Some("couldnt_find_community"), inner: Other errors which are not explicitly handled
Caused by:
0: error sending request for url (https://lemmy.ml/.well-known/webfinger?resource=acct:cryptography@lemmy.ml): error trying to connect: error:16000069:STORE routines:ossl_store_get0_loader_int:unregistered scheme:../crypto/store/store_register.c:237:scheme=file, error:80000002:system library:file_open:reason(2):../providers/implementations/storemgmt/file_store.c:267:calling stat(/usr/lib/ssl/certs), error:16000069:STORE routines:ossl_store_get0_loader_int:unregistered scheme:../crypto/store/store_register.c:237:scheme=file, error:80000002:system library:file_open:reason(2):../providers/implementations/storemgmt/file_store.c:267:calling stat(/usr/lib/ssl/certs), error:16000069:STORE routines:ossl_store_get0_loader_int:unregistered scheme:../crypto/store/store_register.c:237:scheme=file, error:80000002:system library:file_open:reason(2):../providers/implementations/storemgmt/file_store.c:267:calling stat(/usr/lib/ssl/certs), error:16000069:STORE routines:ossl_store_get0_loader_int:unregistered scheme:../crypto/store/store_register.c:237:scheme=file, error:80000002:system library:file_open:reason(2):../providers/implementations/storemgmt/file_store.c:267:calling stat(/usr/lib/ssl/certs), error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:../ssl/statem/statem_clnt.c:1889: (unable to get local issuer certificate)
1: error trying to connect: error:16000069:STORE routines:ossl_store_get0_loader_int:unregistered scheme:../crypto/store/store_register.c:237:scheme=file, error:80000002:system library:file_open:reason(2):../providers/implementations/storemgmt/file_store.c:267:calling stat(/usr/lib/ssl/certs), error:16000069:STORE routines:ossl_store_get0_loader_int:unregistered scheme:../crypto/store/store_register.c:237:scheme=file, error:80000002:system library:file_open:reason(2):../providers/implementations/storemgmt/file_store.c:267:calling stat(/usr/lib/ssl/certs), error:16000069:STORE routines:ossl_store_get0_loader_int:unregistered scheme:../crypto/store/store_register.c:237:scheme=file, error:80000002:system library:file_open:reason(2):../providers/implementations/storemgmt/file_store.c:267:calling stat(/usr/lib/ssl/certs), error:16000069:STORE routines:ossl_store_get0_loader_int:unregistered scheme:../crypto/store/store_register.c:237:scheme=file, error:80000002:system library:file_open:reason(2):../providers/implementations/storemgmt/file_store.c:267:calling stat(/usr/lib/ssl/certs), error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:../ssl/statem/statem_clnt.c:1889: (unable to get local issuer certificate)
2: error:16000069:STORE routines:ossl_store_get0_loader_int:unregistered scheme:../crypto/store/store_register.c:237:scheme=file, error:80000002:system library:file_open:reason(2):../providers/implementations/storemgmt/file_store.c:267:calling stat(/usr/lib/ssl/certs), error:16000069:STORE routines:ossl_store_get0_loader_int:unregistered scheme:../crypto/store/store_register.c:237:scheme=file, error:80000002:system library:file_open:reason(2):../providers/implementations/storemgmt/file_store.c:267:calling stat(/usr/lib/ssl/certs), error:16000069:STORE routines:ossl_store_get0_loader_int:unregistered scheme:../crypto/store/store_register.c:237:scheme=file, error:80000002:system library:file_open:reason(2):../providers/implementations/storemgmt/file_store.c:267:calling stat(/usr/lib/ssl/certs), error:16000069:STORE routines:ossl_store_get0_loader_int:unregistered scheme:../crypto/store/store_register.c:237:scheme=file, error:80000002:system library:file_open:reason(2):../providers/implementations/storemgmt/file_store.c:267:calling stat(/usr/lib/ssl/certs), error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:../ssl/statem/statem_clnt.c:1889: (unable to get local issuer certificate)
3: error:16000069:STORE routines:ossl_store_get0_loader_int:unregistered scheme:../crypto/store/store_register.c:237:scheme=file, error:80000002:system library:file_open:reason(2):../providers/implementations/storemgmt/file_store.c:267:calling stat(/usr/lib/ssl/certs), error:16000069:STORE routines:ossl_store_get0_loader_int:unregistered scheme:../crypto/store/store_register.c:237:scheme=file, error:80000002:system library:file_open:reason(2):../providers/implementations/storemgmt/file_store.c:267:calling stat(/usr/lib/ssl/certs), error:16000069:STORE routines:ossl_store_get0_loader_int:unregistered scheme:../crypto/store/store_register.c:237:scheme=file, error:80000002:system library:file_open:reason(2):../providers/implementations/storemgmt/file_store.c:267:calling stat(/usr/lib/ssl/certs), error:16000069:STORE routines:ossl_store_get0_loader_int:unregistered scheme:../crypto/store/store_register.c:237:scheme=file, error:80000002:system library:file_open:reason(2):../providers/implementations/storemgmt/file_store.c:267:calling stat(/usr/lib/ssl/certs), error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:../ssl/statem/statem_clnt.c:1889:, context: SpanTrace [{ target: "lemmy_apub::fetcher", name: "resolve_actor_identifier", file: "crates/apub/src/fetcher/mod.rs", line: 21 }, { target: "lemmy_apub::api::read_community", name: "perform", fields: "\u{1b}[3mself\u{1b}[0m\u{1b}[2m=\u{1b}[0mGetCommunity { id: None, name: Some(\"cryptography@lemmy.ml\"), auth: Some(Sensitive) }", file: "crates/apub/src/api/read_community.rs", line: 25 }, { target: "lemmy_server::root_span_builder", name: "HTTP request", fields: "\u{1b}[3mhttp.method\u{1b}[0m\u{1b}[2m=\u{1b}[0mGET \u{1b}[3mhttp.scheme\u{1b}[0m\u{1b}[2m=\u{1b}[0m\"http\" \u{1b}[3mhttp.host\u{1b}[0m\u{1b}[2m=\u{1b}[0mcampfyre.nickwebster.dev \u{1b}[3mhttp.target\u{1b}[0m\u{1b}[2m=\u{1b}[0m/api/v3/community \u{1b}[3motel.kind\u{1b}[0m\u{1b}[2m=\u{1b}[0m\"server\" \u{1b}[3mrequest_id\u{1b}[0m\u{1b}[2m=\u{1b}[0mfff82c47-e8f0-4a4b-a475-28a09499a8c3 \u{1b}[3mhttp.status_code\u{1b}[0m\u{1b}[2m=\u{1b}[0m400 \u{1b}[3motel.status_code\u{1b}[0m\u{1b}[2m=\u{1b}[0m\"OK\"", file: "src/root_span_builder.rs", line: 16 }] }
That is because the community hasn’t been synced. In 0.17.4 it was just a generic 404 community not found.
When you are searching are you leaving the filter to all or setting it to community?
Can you post a screenshot of your filter/search query?
Might be taking some time to backfill, I can pull down posts when I explicitly go to the community (see edit 2), but no comments are showing up as of now https://campfyre.nickwebster.dev/post/21
this is me fwiw